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Preface 



This project was supported by Grant No. 96-MU-MU-0008, awarded by the National 
Institute of Justice in cooperation with the Bureau of Justice Assistance, Office of Justice 
Programs, U.S. Department of Justice. NIJ (with help from BJA) is the primary sponsor of this 
research, and the research activities from which we report results here were designed and begun 
under the NIJ grant. As the project progressed, additional support for the work reported was 
provided by the Office of Juvenile Justice Delinquency Prevention, Office of Justice Programs, 
U.S. Department of Justice through Grant No. 98-JN-FX-0004. With OJJDP support we 
expanded data collection activities to include information on juvenile gangs and on activities 
directed at gangs. The project also depended on the support by the Planning and Evaluation 
Service, U.S. Department of Education, of a Study on Violence and Prevention through a 
contract with Westat. The Department of Education was required by Congress to investigate 
violence in schools and its prevention. Westat merged some of its research tasks with those of 
the present project - particularly the student and teacher surveys - in order to maximize 
resources and minimize the burden on schools. Views expressed are those of the authors and do 
not necessarily represent the official position or policies of the U.S. Department of Justice. Nor 
do they necessarily represent the position or policies of other sponsors or organizations. 

The overall design for the project was drafted by Gary D. Gottffedson and fjenise C. 
Gottffedson in a grant proposal submitted to NIJ in August 1995, with subsequent revision 
(including a reduced budget) in November 1995. After NIJ made a grant award in August 1996, 
Gary and Denise Gottffedson began work elaborating a taxonomy of school-based activities to 
prevent problem behavior. We were assisted in this effort by Shannon C. Womer who gathered 
information from federal and state government agencies, foundations, technical assistance 
providers, and others about the range of activity undertaken in schools with the aim of preventing 
or reducing drug use, delinquency, and other forms of problem behavior or to promote a safe and 
orderly school environment. Ms. Womer’s work contributed greatly to the development of the 
taxonomy, which was completed in the early Spring of 1997. 

The taxonomy was the basis for the design of questionnaires to gather information about the 
nature and extent of school prevention activities from school principals - the Phase 1 survey - 
conducted in the spring of 1997. The Phase 1 survey was coordinated by Ellen R. Czeh. She 
was assisted by Suzanne Busby, Rebecca Gold, Elizabeth Jones, Jacob Lawrence, Kirsten 
Mackler, Felicia Morings, and Nicole Piquero who telephoned schools in Herculean efforts to 
extract questionnaire returns. 

While Phase 1 data were being collected, Gary Gottffedson and Denise Gottffedson 
developed the Phase 2 questionnaires. Reviews of school based prevention programs completed 
by Denise Gottffedson (1997, in press) and the taxonomy were important sources of guidance in 
developing the Phase 2 Principal Questionnaire and the fourteen distinct Activity Coordinator 
Questionnaires. The principal questionnaire adapted measures of conscientiousness from the 
work of Goldberg (1992) and of accomplishment record from the work of G. Gottffedson (1994). 

iii 
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The Phase 2 Student Questionnaire was adapted from the Effective School Battery student survey 
(G. Gottfredson, 1984/1999), What About You (G. Gottfredson & D. Gottfredson, 1992, 1999), 
and the School Action Effectiveness Study student questionnaire (G. Gottfredson, 1982) with 
new material based on the taxonomy developed for this project. The Phase 2 Teacher 
Questionnaire was adapted from the Effective School Battery teacher survey (G. Gottfredson, 
1984/1999), and the Organizational Focus Questionnaire (G. Gottfredson and Holland, 1997), 
with the incorporation of original material based on the taxonomy. Ellen Czeh assisted in the 
production of the seventeen separate questionnaires developed for Phase 2. 

Sally Hillsman, Thomas Feucht, Rosemary Murphy, and Winifred Reed of the National 
Institute of Justice (DOJ) and Joanne Wiggins of the Planning and Evaluation Service (ED) 
worked to develop the Memorandum of Understanding between PES and NIJ to share data and 
data collection instruments that had been developed for Phase 2 surveys, which was signed by 
Alan L. Ginsburg, Director of PES, and Jeremy Travis, Director of NIJ. Following this, Scott 
Crosse and Irene Hantman of Westat worked with Joanne Wiggins of PES to obtain Office of 
Management and Budget clearance that would be necessary for teacher and student surveys to be 
collected by Westat under contract with ED, and they suggested minor revisions in questions. 
David Cantor of Westat suggested additions to the Phase 2 principal questionnaire to capture 
school crime data similar to that captured in other surveys. 

Data collection responsibilities for Phase 2 were divided between Gottfredson Associates 
and Westat, with Westat focusing on secondary schools where teacher and student surveys would 
be conducted and Gottfredson Associates focusing on elementary schools. Scott Crosse was 
study director for the Westat effort. Irene Hantman led the data collection effort at Westat. She 
was assisted by Katie Andrew, Julie Anderson, Betty Barclay-Hurley, Kristen Heavener, Robin 
Hill, Galen McKeever, Pat McClure, Sheri Nicewamer, Parvis Omidpanah, Jeff Roussos, and 
Fran Winter in recruiting schools and by Liv Aujla, Kevin Jay, Steve Linz, Kim Standing, and 
Diane Steele in data collection. She was also assisted by A1 Bishop, John Brown, Jason Grim, 
and Ying Long in data management. Ellen Czeh led the data collection effort at Gottfredson 
Associates. She was assisted by Rebecca Silverman and Adriana Wade who co mm u n icated with 
schools to secure the return of data, and by Nisha Gottfredson and Kara Czeh who prepared 
survey materials. 

Gary Shapiro and Lana Ryaboy of Westat developed nonresponse weights that were used 
together with initial sampling weights to produce national estimates reported here, and they 
advised Gary Gottfredson on the calculation of sampling errors. Elizabeth Jones prepared initial 
data files from survey data and performed initial psychometric analyses for student and teacher 
data. Ratings of prevention activity quality were devised by Denise Gottfredson and Gary 
Gottfredson, and psychometric analyses for discretionary activity data were performed by Denise 
Gottfredson and for Principal data by Gary Gottfredson. Allison Payne performed yeoman 
service in coding the complicated information provided by activity coordinators, April Simonsen 
prepared census data for schools, and Shawn Anderies coded information principals provided in 
Activity Detail Booklets to produce the measures of span of control and delegation. Statistical 




iv 

6 



analyses reported here were prepared by Gary Gottfredson and Denise Gottfredson. Ellen Czeh 
assisted in the preparation (over and over again) of tables. 

We are grateful for the endorsement of the project by Thomas F. Koemer, Deputy Executive 
Director of the National Association of Secondary School Principals, and by Ronald J. Areglado, 
Associate Executive Director for Programs of the National Association of Elementary School 
Principals. Letters from these association leaders and a letter from Jeremy Travis, Director of the 
National Institute of Justice, assisted in encouraging school principals to participate in the 
project. 

The report was written by Gary Gottfredson and Denise Gottfredson, who were assisted by 
Ellen Czeh. 
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Introduction to the Study 



The National Study of Delinquency Prevention in Schools (NSDPS) was undertaken to 
develop a comprehensive account of the levels of problem behavior in United States schools and 
of what schools do to prevent problem behavior and to promote a safe and orderly environment. 
In this first major report from the study, we aim to provide a description of the full range of 
activities schools undertake to reduce or prevent problem behavior — including delinquency, drug 
use, and violence. 

The study contrasts sharply with much evaluation research that is directed at assessing the 
effectiveness of specific practices. Evaluation is sometimes defined as activity to learn what was 
done, how, and with what effect. But the present research was not undertaken to assess the 
effectiveness of specific instances of prevention or intervention activities. Much evaluation 
research examines isolated programs or a circumscribed set of activities or arrangements and 
seeks to determine their effects. Good contemporary evaluation research usually also assesses 
the strength and integrity of program implementation (Sechrest, West, Phillips, Redner, & 
Yeaton, 1979). But many program evaluations and most instances in which evaluators measure 
the quality (strength and integrity) of program implementation lack ecological validity 
(Brunswick, 1947). Because the research is designed to focus on one or a small number of 
specific realizations of a program or practice, it lacks a sufficiently representative design to 
describe typical practices or the typical degree of strength and integrity attained when programs 
are applied outside of the experimental context. In contrast, the present research was designed to 
assess the nature, extent, and quality of prevention and intervention activity directed at problem 
behavior and school safety in a representative sample of the nation’s schools. 

Growth in Development of Prevention Programs 

Recent years have seen growth in the development and application of prevention programs 
— most of these directed at adolescents and based in schools, but some directed at other groups. 
Wilson-Brewer et al. (1991) identified 83 violence prevention programs in 20 states. They 
obtained survey responses from 51 of these programs, and the data indicate that most of these 
had been initiated recently. These programs had multiple sources of support: Most were funded 
by foundations (52%), and many operated on fee-for-service (44%), state funds (34%), federal 
funds (32%), or city sponsorship (30%). Most of these programs reached the target populations 
(typically adolescents and young adults) indirectly by working with teachers (41%), school 
administrators (32%), and a variety of other intermediaries. Middle and high schools were the 
predominant loci of the programs (62% of programs in each of these school settings). Only 21% 
reported any type of outcome evaluation; even counts of individuals affected were relatively rare. 

A large number of programs directed at alcohol, tobacco and other drug use and more 
recently at violence have been sponsored by the Center for Substance Abuse Prevention (CS AP, 
1994). Between October 1987 and September 1994 CSAP made 363 grants directed at high risk 
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youths, mostly (56%) to not-for-profit organizations and 1 1% to educational systems (although 
many more of these programs operate in or are focused on schools). 

The Bureau of Alcohol, Tobacco and Firearms (BATF) promotes and sponsors GREAT 
programs; the Administration on Children, Youth and Families (ACYF) sponsors a major youth 
gangs and drug prevention program; important initiatives are sponsored by the National Institute 
for Child Health and Development, the Department of Education, the Centers for Disease 
Control and Prevention, the Office for Juvenile Justice and Delinquency Prevention; and research 
and demonstration programs are supported by the National Institute on Drug Abuse and other 
Institutes. In addition, many foundations are involved in supporting programs to prevent 
problem behavior. Among them: Arizona Community Foundation, Bell of Pennsylvania, Best 
Foundation, CAP Cities/ABC, Eisenhower Foundation, Foundation for New Era Philanthropy, 
Foundation for the National Capital Region, Goldseker Foundation, Grantmakers in Health, GTE 
Corporation, Hogg Foundation, IBM, J.M. Foundation, Robert Wood Johnson Foundation, Henry 
J. Kaiser Family Foundation, Weing Kaufman Foundation, Kellog Foundation, Nathan 
Cummings Foundation, National Masonic Foundation, New York Community Trust, Okura 
Foundation, Pew Charitable Trust, Pool Health Care Trust, Santa Clara Community Partnership, 
Kansas Health Foundation, Winston-Salem Foundation. 

Continued growth in these programs may be expected in part because national reports have 
directed attention to their importance and called for further development. One of the national 
education goals is directed at increasing safety (Office of Educational Research and 
Improvement, 1 993). Healthy People 2000 (U.S. Department of Health and Human Services, 
1991) called for teaching conflict resolution skills in half the nation's schools by the year 2000. 
The National Institute for Child Health and Human Development has sponsored developmental 
efforts on adolescent decision making to reduce violence and other risky youth behavior (Baron 
& Brown, 1991). And the National Institute of Justice Program Plan for recent years has called 
attention to school-based prevention programs. 

School as a locus of intervention. The school is a key locus for intervention not simply 
because adolescents spend so much time there. It is the primary institution aside from the family 
that has access over extended periods of time to most of the population of young people (G. 
Gottfredson, 1981, 1987a; Martin et al., 1981). Until school dropout becomes a major problem 
(mostly after grade 9), this access is almost universal. Despite complaints that the schools cannot 
be expected to do everything and some persons' views that schools ought not have roles in 
socializing the young beyond narrow educational bounds, the school offers a realistic opportunity 
for delivering interventions to reduce delinquency. The reality of programming directed at 
youths is that the lion's share of money spent by government agencies on children and youths is 
spent on education - probably upwards of 85% in the states and about 42% of federal spending 
(Holmes, Gottfredson, & Miller, 1992). 
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School-Based Interventions 



School-based prevention programs take on a number of distinguishable forms. Although 
few programs resemble a pure type, some of the more prominent kinds of interventions believed 
to have potential are (1) social competency programs directed at high-risk individuals or at the 
general population of adolescents, (2) behavior management programs whether applied in the 
school or through the involvement of parents, (3) programs directed at environmental change to 
increase the effectiveness of school management or behavior management in schools, (4) 
programs to increase the bonding of individuals to the social order, (5) programs to exclude 
weapons or intruders from school, or limit the availability of weapons, (6) programs to improve 
opportunities for surveillance, (7) programs to provide recreation or productive youth activity, 
and (8) programs that provide information. Each of these types is discussed briefly in turn in the 
following paragraphs. Actual programs generally combine features of more than one ideal type, 
so that many social competency programs also include components that provide information, 
many programs contain recreational elements, and so on. 

Social Competency Programs 

One set of programs known as "social competency" interventions is directed at self-restraint. 
These are also often called cognitive-behavioral interventions. Social competence programs 
generally involve: (a) developing people's skills in identifying the antecedents of problems in the 
cues they perceive from others, their environment, and their own state of arousal, (b) increasing 
the probability that people will hesitate before taking impulsive action, (c) improving individuals' 
capacity to process information with reference to the desirability of alternative outcomes, and (d) 
establishing behavioral repertoires for coping with events with potential to lead to harm. Some 
of these programs involve parent training to help them teach cognitive behavioral self- 
management to their children (e.g., Spivak and Shure's, 1976, Interpersonal Cognitive Problem 
Solving or Camp and Bash's, 1985, Think Aloud program); others are administered by teachers 
(e.g., Botvin's, 1989, Life Skill Training or the Weissberg et al., 1990, Social Problem Solving 
Program). (See Elias et al., 1994. See also Baron & Brown, 1991.) These programs are most 
effective when they teach social competency content using behavioral strategies such as rehearsal 
and role-playing (D. Gottfredson, Wilson & Najaka, in press). 

Single-project evaluation research has demonstrated that social competency promotion 
programs that make use of high levels of modeling and practice, provide specific and frequent 
feedback about new behaviors, provide cues to prompt the behavior, and use techniques to 
generalize the new behavior to different settings can reduce crime (Arbuthnot & Gordon, 1986; 
Arbuthnot, 1992; Shapiro & Paulson, 1998; Tremblay et al., 1991; Tremblay et al., 1992; 
Tremblay et al., 1994; Tremblay et al., 1995; McCord et al., 1994) and substance use (e.g., 
Kaufman et al., 1994; Botvin et al., 1990; Botvin, Baker, Renick, et al., 1984; Botvin, Baker, et 
al., 1995; Shope, Copeland, Marcoux, & Camp, 1996; Caplan et al., 1992). They can also work 
to reduce anti-social behavior and other conduct problems (e.g., Amerikaner and Summerlin, 
1982; Elkin et al., 1988; Feindler et al., 1984; Conduct Problems Prevention Research Group, 
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1999a, 1999b; Coie, 1997; Shure & Spivack, 1979, 1980, 1982; Weissberg & Caplan, 1994). 
These interventions have been shown to be efficacious in trials with pre-school (Shure & 

Spivack, 1979, 1980, 1982), elementary (Amerikaner & Summerlin, 1982; Conduct Problems 
Prevention Research Group, 1999a; Coie, 1997; Elkin et al., 1988; Gesten et al., 1982; Gesten et 
al., 1979; Greenberg et al., 1995; Hudley, 1994; Pepler et ah, 1991; Weissberg, Gesten, Rapkin, 
et ah, 1981), junior high (Botvin et ah, 1990; Botvin, Baker, Renick, et ah, 1984; Botvin, Baker, 
et ah, 1995; Ellickson & Bell, 1990; Ellickson et ah, 1993; Kaufman et ah, 1994; Shope, 
Copeland, Marcoux, & Kamp, 1996; Caplan et al., 1992; Feindler et ah, 1984; Weissburg & 
Caplan, 1994), and senior high (Arbuthnot & Gordon, 1986; Arbuthnot, 1992; Hecht et ah, 1993; 
Sarason & Sarason, 1981; Eggert et ah, 1990; Severson et ah, 1991; Shope, Copeland, Maharg, 

& Dielman, 1996) students. Social competency promotion programs can be applied to the 
general population or to a targeted subpopulation of high-risk individuals. Meta-analyses 
(quantitative synthesis of evidence from many studies) imply that effective delinquency programs 
often incorporate cognitive-behavioral approaches to developing social competencies (Izzo & 
Ross, 1990; Lipsey, 1992). 

Behavior Management Programs , 

A well developed technology exists for intervening with individual youths who display 
impulsive, aggressive, or conduct disordered behavior (Kazdin, 1987). A logical extension of 
such effective behavioral methods is their application in classrooms and schools. Research on 
classroom management documents effective practices (Brophy, 1983; Doyle, 1986; Emmer & 
Aussiker, 1989; Evertson & Harris, 1992; D. Gottffedson, 1992a). Similarly, schools.can 
involve parents in behavior management, including home-based backup reinforcement for school 
behavior (Atkeson & Forehand, 1979; Barth, 1979) and programs to provide parents with 
training in behavior management (Dishion & Andrews, 1995). Bry (1982) and Bry and George 
(1979, 1980) have demonstrated a behavioral program directed at tardiness, class preparation and 
performance, behavior and attendance in which students earned points contingent on their 
behavior using trips for a backup reinforcer. Bry and George’s intervention improved behavior 
after students had been exposed to the intervention for two years and positive effects were found 
five years after the program ended. 

The same principles can be applied to entire classrooms. A Good Behavior Game (a group 
contingency management program developed by Barrish, Saunders & Wolf, 1969) has repeatedly 
been shown to be efficacious in reducing disruptive behavior misconduct (Barrish et al., 1969; 
Bostow & Geiger, 1976; Darveaux, 1984; Fishbein & Wasik, 1981; Grandy, Madsen, & De 
Mersseman, 1973; Harris & Sherman, 1973; Hegerle, Kesecker, & Couch, 1979; Johnson, 

Turner, & Konarski, 1978; Kosiec, Czemicki & McLaughlin, 1986; Medland & Stachnik, 1972; 
Phillips & Christie, 1986; Swiezy, Matson, & Box, 1992; Warner, Miller, & Cohen, 1977) and 
aggressive behavior (Dolan et al., 1993; Huber, 1979; Saigh & Umar, 1983) in elementary 
classroom, preschool, library, and a comprehensive school for slow-leaming disruptive students. 
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Environmental Competence in Guardianship 

A variety of interventions are directed at enhancing the capacity of school and other 
environments to signal appropriate and inappropriate behavior or to improve mechanisms for 
watching for and responding to student behavior. 

Defining norms. One impressive line of research and demonstration to limit conflict in 
schools has been undertaken in Norway (Olweus, 1991, 1992a; Olweus & Alsaker, 1991). 

Olweus noted that certain adolescents, called "bullies," repeatedly victimized other adolescents. 
Typical bullies were characterized as displaying an "aggressive reaction pattern combined (in the 
case of boys) with physical strength" and as representing "a more general conduct disordered, 
antisocial and rule-breaking behavior pattern." Olweus also noted that the victims of bullying 
tended to be neglected by the school. Although they were known to be targets of harassment, the 
problem was largely ignored by adults who failed to actively intervene and thus provided tacit 
acceptance of the bullying. 

A program was devised based on the notion that, "Every individual should have the right to 
be spared oppression and repeated, intentional humiliation, in school as in society at large." The 
campaign directed communication to redefining bullying as wrong. A booklet was directed to 
school personnel, defining the problem and spelling out ways to counteract it. Parents were sent 
a booklet of advice. A video illustrating the problem was made available. And questionnaire 
surveys to collect information and register the level of the problem were fielded. Information 
was fed back to personnel in 42 schools in Bergen, Norway. Reassessment implied considerable 
diminution in the problem - results consistent with an interpretation that the environments had 
become more competent in establishing norms as a result of the campaign. 

School-wide capacity-building or behavior management. The application of behavior 
management programs on a school-wide basis is a form of environmental competency 
enhancement. But the employment of these methods is not straightforward. Schools and school 
systems generally have guidelines for school personnel in the form of discipline codes and 
reactive strategies, and evidence shows that variations in school discipline practices are indeed 
related to levels of victimization in schools (G. Gottfredson & Gottfredson, 1985). But, most 
violence occurs in urban schools serving relatively high crime, disorganized, and high proportion 
minority populations, and in schools which themselves suffer problems of low staff morale and 
difficulty in recruiting and retaining first-rate personnel. Problem schools are often overwhelmed 
by problems, despite the heroic efforts of educators to cope with them (Emmer, 1992; G. 
Gottfredson, 1987b). 

Effective programs to reduce disorder have, nevertheless, been demonstrated in schools with 
multiple problems. In one of these (D. Gottfredson, 1988), a structured organization 
development method (Program Development and Evaluation; PDE; G. Gottfredson, 1984a; G. 
Gottfredson, Rickert, Gottfredson, & Advani, 1999) was applied in a three-year effort to reduce 
disorder in a troubled Baltimore City school. The program designed, implemented, and refined 
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interventions to increase the predictability of responses to students' disciplinary infractions, 
increase rewards for appropriate behavior, and increase prosocial peer and teacher support. The 
program was effective in reducing disorder. 

The PDE method - in which researchers work with school personnel to define goals and 
objectives, develop program theory, plan for and monitor the implementation of program design 
choices, and assess outcomes - was also applied in programs in seven secondary schools (D. 
Gottfredson, 1986). District personnel used PDE to develop a general plan and then used the 
PDE method to make school-specific plans for school improvement and implementing 
interventions. The effort increased the clarity and consistency of school rules, student success, 
and attachment; and it reduced problem behavior as well as staff morale and other indicators of 
school capacity. 

In another study, eight schools participated in a program to increase the clarity of school 
rules and to promote their application in a fair, firm, and consistent way (D. Gottfredson, 
Gottfredson, & Hybl, 1993). Again, in the context of an organization development framework, 
extensive administrator and teacher training was coupled with the development of school 
mechanisms for attending to and responding to student behavior using guidelines for teacher and 
administrator responses. Teachers were trained to use effective classroom organization and 
management techniques. Computerized behavior tracking was used to promote the clarity and 
consistency of responses to student behavior. Evaluation showed that the program's effectiveness 
differed from school to school in approximate proportion to the quality of program 
implementation, and it was effective in reducing conduct problems in. high implementation, 
schools. 

Related approaches to reducing problem behavior on the way to and from school have been 
attempted in several places. Kenney and Watson (1996) engaged students in applying a four-part 
planning method (SARA) often recommended for use in community-oriented policing. Students 
identified safety problems and proposed methods to ameliorate them. Reductions in student fear 
were observed. G. Gottfredson, Gore, & Jones (1998) engaged school faculty and students in 
planning to prevent problem behavior in improve attendance in a very disorganized school. The 
approach to planning was simple and low key to overcome resistance to more formal approaches 
to planning. After two years, attendance rose about 5% above historical levels and teacher 
morale and perceptions of safety improved, although the school remained very disorderly. 

Other Interventions 

Increasing Bonding. Prevention programs have applied a number of intervention models 
apparently aimed at increasing social bonding. This has included use of cooperative learning 
techniques (Johnson & Johnson, 1989; Slavin et al., 1990) to increase rewarding academic 
experiences and liking for school, mentors to provide positive role models and prosocial adults to 
whom youths may become attached (Hahn, Leavitt, & Aaron, 1994; LoSciuto et al., 1996), field 
trips to the community and discussions of laws and social problems as part of some forms of law- 
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related education (D. Gottfredson & Gottfredson, 1992). Other programs involve scholastic 
goal-setting and incentives for improved performance (Mac Iver, 1993). 

A number of culture-specific programs, such as Afro-centric rites of passages programs and 
programs to instill a sense of awareness and pride in cultural roots or traditions may be regarded 
as bonding programs that promote values education and attachment to a social group. 

Excluding weapons and intruders. A number of approaches to reducing crime have focused 
on mechanisms to limit access to schools by intruders or to prevent weapons from coming into 
schools (Butterfield & Turner, 1989). A range of approaches are used, including efforts to 
control entry into schools through the use of checkpoints and identification systems, metal 
detectors, and security patrols or officers who challenge intruders (Quarles, 1989; Gaustad, 

1991). Sometimes school doors are fitted with electromagnetic locks that open when a fire alarm 
is set off. Although such programs are controversial, some experience implies that they can be 
effective and these are worthy of more systematic tests (Aleem & Moles, 1993). 

Improving opportunity for surveillance. Some schools are designed in a manner that makes 
it easy to observe who enters the building and what people in the building are doing. Other 
schools, including many older urban schools, employ architecture that makes observation 
difficult. When school design makes surveillance difficult, some schools are retro-fitted with 
video cameras to monitor hallways, stairs, and entrances and with “panic bars” on exit doors so 
that an alarm is triggered if a door is opened from the inside. In extreme cases, portions of 
school buildings are physically walled off so that no one can enter areas that are difficult to 
monitor. To the best of our knowledge, there have been no formal evaluations of these 
approaches, but taking steps to improve opportunity for surveillance are plausible methods for 
improving school safety. 

Recreation and youth employment. Recreation programs include regular after-school 
recreation programs with or without an instructional component, police athletic leagues, safe 
haven programs, and late night recreation programs. Programs to employ youths during the 
summertime are also generally intended to provide constructive activity. Sometimes these purely 
recreational or employment programs are combined with program elements of another program 
type, which increases their plausibility and delinquency prevention potential. Often, a rationale 
for recreation programs is that they provide supervision for youths who would otherwise be 
unsupervised in after-school hours. D. Gottfredson (1997) reviewed the evidence about 
alternative or recreational activities and concluded that there is little reason to believe that typical 
recreation programs will be helpful in reducing delinquency and that they have the potential to 
increase it if they bring high-risk youths together. 

Information Programs. At one time, many drug prevention programs were primarily 
informational in nature. The provision of information is still a part of most drug prevention 
programs, and a few programs are still almost exclusively informational in nature. Crime 




1-7 



s. 



17 



prevention programs that provide information about the conditions under which crime occurs so 
that citizens can take steps to limit their exposure to risk remain common. 

Hybrid Programs. Most prevention programs are hybrids in the sense that they combine 
elements that resemble two or more of these ideal program types. For example, Botvin's Life 
Skills Training program (Botvin et al., 1984) is mainly a social competence program, but it 
includes a large segment that is informational. The Drug Abuse Resistance Education (DARE) 
curriculum (Bureau of Justice Assistance, 1988) implemented by police officers is very widely 
applied and highly regarded (Police Research Center, 1995; Ringwalt & Greene, 1993) program 
directed at enhancing upper elementary children's social skills, particularly in recognizing and 
resisting peer influence to use drugs; and it also focuses on drug information, decision skills, and 
self-esteem - making it a hybrid program. Another example of a school-based program that 
mixes some social competency training with drug information is the Project ALERT curriculum 
disseminated by the Best Foundation (1993). Pentz et al. (1990) have employed multiple 
methods (including parents and the media) directed at adolescent social skills. The GREAT 
programs are also hybrids, similar to DARE. An ambitious approach to gang suppression and 
intervention (Spergel, 1 990) is another hybrid program that emphasizes mobilizing communities 
to improve their safety and protect others, utilize environmental design techniques to enhance 
guardianship, and take other steps. A critical element in the model is a special focus on 
providing safe, gang-free schools by involving key individuals in and out of the school to 
improve guardianship. 

A great many things can potentially be done in schools by those who seek.to reduce, or 
prevent problem behavior. Some of these things have been the object of scientific study. Others 
have not. A series of recent reviews and summaries (Botvin, 1990; Brewer, Hawkins, Catalano, 
& Neckerman, 1995; Eron, Gentry, & Schlegel, 1994; Hansen, 1992; Hansen & O’Malley, 1996; 
Hawkins, Arthur, & Catalano, 1995; Institute of Medicine, 1994; Schinke, Botvin, & Orlandi, 
1991; Tobler, 1992; Weissberg & Greenberg, 1997) attest to the potential of preventive 
interventions, leading to the optimistic slogan “prevention works.” The recent reviews by D. 
Gottfredson (1997, 2000) are somewhat more circumspect about the broad, potential of 
preventive interventions to reduce problem behavior and drug use but also illustrate the potential 
of these interventions. 

The Problem of Implementation 



Wilson-Brewer et ah's (1991) survey of violence prevention programs identified four main 
areas of barriers to success: (1) Almost all programs had difficulty securing sufficient and stable 
funding to acquire staff, operate programs of significant scale and duration, and maintain 
continuity over time. (2) Half of programs working with school systems faced overworked, 
stressed, and bumed-out teachers. When school personnel are asked to implement a program 
they have not selected, they feel overburdened with work, or they do not perceive support for 
programs of sufficient scale, they resist implementation. (3) Programs - especially those 
involving gang activity - saw denial of the existence of serious safety problems (despite clear 
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problems) as a barrier to effective programs. (4) About a third of programs lacked the expertise, 
money, or assistance to evaluate their activities. 

To these obstacles may be added those identified by Elias et al. (1994) in their review of 
competence promotion programs: factors related to the readiness of organizations to implement 
change. As they put it, "A program consisting of potent and validly conceived mechanisms and 
processes may not succeed because the host environments are not able to support those processes 
(Zins & Ponti, 1990)" (p. 24). Among the factors facilitating or hindering implementation are 
organizational climate and norm structure, the organization’s history of response to innovations, 
the balance of new and experienced administrators, the articulation of goals with the programs, 
staff morale, administrator leadership and communication, role definitions, educator involvement 
in planning, and staff resentment of troublesome students (Corcoran, 1985; G. Gottfredson & 
Gottfredson, 1985, 1987). Each factor may facilitate or hinder implementation; if morale is high, 
implementation is easier to achieve, if low, it is harder. 

Implementation of effective prevention efforts is likely to be most difficult in schools and 
communities in which rates of crime, delinquency, and school disorder are greatest. In such 
places morale - a sense that members of the community can count on each other to achieve goals 

- may be low and problem responses may be focused on responses to crises or immediate 
problems rather than on diagnosing problems and planning solutions. In disorganized schools or 
communities, organizational obstacles may thwart the implementation of efficacious strategies 
with sufficient strength and fidelity, and the organizations may fail to improve implementation 
overtime. 

The barriers that prevention programs face can be put in context by recalling that problems 
of implementation have plagued programs in crime and delinquency for decades. The 
bibliography of literature on the rehabilitation of criminal offenders by Lipton, Martinson and 
Wilks (1975) is best remembered by many for the generalization in Martinson's (1974) summary 
"With few and isolated exceptions, the rehabilitative efforts that have been reported so far have 
had no appreciable effect on recidivism." The Lipton et al. review was not alone among 
disappointing reviews (Whitehead & Lab, 1989; Wright & Dixon, 1977). 

But the conclusion that "nothing works" was not a correct conclusion to draw from this 
literature. The National Research Council Panel on Research on Rehabilitative Techniques 
(Sechrest, White, & Brown, 1979) noted that flaws in evaluation methods and - more important 

- limitations in the strength and fidelity of implementation of programs do not justify the 
conclusion that effective programs cannot be applied. Lipsey (1992) conducted a meta-analysis 
of 443 juvenile delinquency treatment programs to examine the relation of program 
characteristics, subject characteristics, researcher characteristics, and evaluation design to 
program effects. Lipsey found that effects overall were small, but that the "dosage" of treatment 
program and features of the treatment program itself were associated with the size of effects. 
More structured, behavioral, and multimodal treatments were more effective. Lipsey's "dosage" 
is equivalent to strength of implementation and his other findings about structure and 
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implementer characteristics suggest fidelity of implementation to a program plan. Lipsey and 
Wilson (1998) examined a subset of studies involving more serious delinquents and found that 
duration of treatment, integrity of treatment implementation, program age, and involvement of 
mental health treatment personnel were predictive of size of interventions’ effects. 

These issues of strength and integrity of program implementation are bound to influence the 
effectiveness of school-based prevention programs as well. Prior research on this topic implies 
that the most important initial question to be answered in an evaluation of school-based 
prevention programs is not "what works?" but "what was done?" 

Evaluations Probably Overestimate the Effectiveness of Interventions 

Evaluations and other research has shown that some kinds of interventions to reduce 
problem behavior can be effective. For example, we cited evidence earlier that behavioral and 
cognitive behavioral interventions have repeatedly been shown to be effective in reducing 
problem behavior or improving attendance. In many cases, however, the evidence derives from 
optimal or at least good implementations of the intervention in question. Often investigators 
train implementers, monitor their behavior, correct implementation errors, or are directly 
involved in the application of the method being studied. In some cases, the evidence is derived 
from schools that were especially amenable to program implementation. For example, the 
developer and principal evaluator of one popular instructional program routinely requires that 
80% of faculty vote to adopt the program by secret ballot before the program will be attempted in 
the school (Jones,. Gottfredson & Gottfredson, 1997; Mathews, 1999; Walberg & Greenberg, . . 
1998). This location selection bias in evaluations of this program, named “Success for All” by 
its developers is not emphasized in their descriptions of it (Slavin, Madden, Dolan, & Wasik, 

1 996) who titled their recent account Every Child, Every School: Success for All. 

In research or demonstration programs, the capacity of the school to serve as an 
implementation site is likely to be greater than the typical school - evidenced at least in part by 
its willingness to participate in a research project. In addition, the particular implementers 
(teachers or others) are likely to be selected for their willingness to implement a program, 
cooperate with evaluators, and their ability quickly to learn to put new methods in place. In all of 
these respects, they are likely to produce better instances of implementation than would be 
achieved in the average school, let alone schools where many youths are engaged in high levels 
of problem behavior or where faculty are demoralized. 

Schools and their personnel differ in the extent to which they are able or willing to produce 
strong and faithful implementations of intended programs. For example, Botvin, Batson, et al. 
(1989) reported variation in the quality of implementation across teachers in an experiment in 
nine urban schools. In another study of eight urban schools, Botvin, Dusenbury, et al. (1989) 
reported that the amount of Life Skills Training material covered by teachers ranged from 44% to 
83%. Positive effects of the program were found only for a high implementation group (with a 
mean completion rate of 78%), not for the low implementation group (mean of 56% delivery). In 



a third study by Botvin et al. (1990), coverage of the curriculum ranged from 27% to 97%, with 
75% of students exposed to 60% or more of the material. The level of implementation was 
strongly related to the effectiveness of the intervention. 

Health and mental health researchers refer to the distinction between intervention efficacy 
(an efficacious intervention can work) and effectiveness (how well the intervention does work 
when applied in typical settings by typical practitioners). In this language, some interventions to 
reduce or prevent problem behavior have been shown to have efficacy, but almost no 
interventions have been shown to be generally effective. If efficacious interventions are 
ineffective, it is likely that flawed implementation is a large part of the reason. 

Hypothesized Factors Leading to Successful Program Implementation 

The National Study of Delinquency Prevention in Schools (NSDPS) was designed to allow 
an examination of the following categories of factors as potential explanations of the successful 
implementation of prevention programs: 

Organizational capacity. Organizational capacity means the capacity of the school to 
implement strong programs. This includes, but is not limited to, the school’s capacity to 
implement delinquency programs or arrangements to promote a safe environment. Our 
conception of organizational capacity is general, and schools lacking organizational capacity are 
expected to have difficulty implementing sound instructional programs of all types, to have 
difficulty "marshaling parental and- staff support for innovations, and projecting a competent, 
effective image to the community. Elsewhere (G. Gottffedson & Gottffedson, 1987) we have 
referred to the limited infrastructure for program development in a school with limited 
organizational capacity. Limited organizational capacity is indicated by poor staff morale, a 
history of failed programs or other innovations in the past, and a sense of resignation about the 
possibilities for improving the school. Experience implies that when schools score low on the 
Morale scale of the Effective School Battery (G. Gottffedson, 1 999), improvement programs are 
difficult to implement. Reviews of factors associated with implementing and sustaining 
innovations (Berman & McLaughlin, 1978; McLaughlin, 1990), evaluations of school-team 
approaches to reducing school crime (Social Action Research Center, 1979, 1980), and our own 
work (D. Gottffedson, Gottffedson, & Hybl, 1993; D. Gottffedson & Gottffedson, 1992; D. 
Gottffedson et al., 1998; G. Gottffedson, 1982; G. Gottffedson, Gottffedson, & Cook, 1983; G. 
Gottffedson & Gottffedson, 1987) on implementing and evaluating delinquency prevention 
programs and programs to manage student behavior all imply that organizational capacity is 
important for implementation. 

Turnover in personnel or unpredictability in staff responsibilities is expected to undermine 
the orderly execution of many school functions, including the application of activities to promote 
a safe and orderly environment and other prevention activities. Turnover is related to 
expectations or intentions to quit a work environment and to organizational commitment 
(Mobley, Griffith, Hand, & Meglino, 1979; Porter & Steers, 1973; Porter, Steers, Mowday, & 
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Boulian, 1974), and so organizations with high levels of turnover may have more difficulty 
implementing high quality prevention activities not only because of the direct effects of 
instability in staffing but also because of the organizational climate concomitants of turnover. 

Leadership and staff traits and past accomplishments. Leadership means orienting a group 
towards goals and objectives; providing incentives, feedback, and supervision to further those 
goals and objectives; arranging the support needed and removing obstacles; and planning the 
steps and arrangements necessary to move towards goals. Research on leadership implies that 
two (initiating structure and consideration; Fleishman, & Harris, 1962) or more (Clark & Clark, 
1990; G. Gottfredson & Hybl, 1987; Yukl & Van Fleet, 1992) dimensions are useful in 
describing leadership behavior. Educational research implies that the leadership of a principal or 
of another responsible party in a school is important in improving educational programs (Hall, 
1987; Hall, Hord, Huling, Rutherford, & Stiegelbauer, 1983; Hord, 1981). Workers’ general 
ability has been found to be a robust predictor of quality of work performance across a wide 
range of occupations (Schmidt & Hunter, 1998; Schmidt, Ones & Hunter, 1992), measures of the 
ability or literacy of teachers are important predictors of test score gains (Ferguson, 1991; 
Ferguson & Lad, 1996) in studies in two states and in a recent meta-analysis (Grenwald, Hedges, 
& Laine, 1996) . Another personality trait, conscientiousness, has also been identified as a 
relatively robust noncognitive predictor of performance across a broad range of occupations 
(Sackett and Wanek, 1996). Conscientiousness is one of five broad personality dispositions 
helpful in summarizing information about personality (Digman, 1990; Goldberg, 1992). 

Workers high in conscientiousness are dutiful, organized, and dependable. Finally, G. 
Gottfredson (1994) has shown that an inventory of the past accomplishments of school principals 
distinguishes those who have been identified by their professional organizations as outstanding 
achievers. Accordingly, leadership behaviors, traits and past accomplishments of leaders or 
program implementers are expected to be related to quality of program implementation. We 
have not attempted to measure general ability in the NSDPS because we assumed that principals 
would regard this as intrusive and reduce response rates in our surveys. But we have measured 
leadership behaviors and conscientiousness. 

Budget and resources. Adequate funding and other resources are presumably required for 
the successful implementation of any intervention. This includes budget support for such thing s 
as materials needed, payment of workers, transportation, or supplies. Presumably it is not total 
budget resources allocated to education or to a school that is required for the successful 
implementation of specific preventive interventions. Instead, what is required may be resources 
available for that specific intervention or the control over money or resources by those who 
operate the program or activity, so that it can be allocated in needed ways. The availability of 
needed resources should facilitate implementation and their lack thwart it. 

Organizational support - training, supervision and support. Most of today’s state of the art 
approaches to the prevention of problem behavior were not a part of the pre-service training of 
many of today’s educators. Some approaches, such as the use of behavioral techniques, have 
been understood for many years. But even such established methods were not always included in 
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the curriculum of teacher training institutions in previous decades. More recent methods, such as 
cognitive-behavioral training and an emphasis on normative expectations for behavior, are less 
likely to have been a part of the preparation of most of elementary and secondary educators now 
working. Accordingly, it is to be expected that training of school personnel will be necessary for 
the implementation of a variety of preventive interventions in schools. The quality of 
implementation will probably depend on the extensiveness and quality of training. Quality of 
training is assumed to include features such as the use of behavioral modeling (Goldstein & 
Sorcher, 1973; Sorcher & Goldstein, 1972) methods, opportunities to anticipate and resolve 
obstacles to application of the method, and follow-up training or coaching. 

Supervision and support are facets of leadership behavior that are important components of 
organizational support. Supervision provides direction for worker behavior when workers 
require direction; and it provides coaching, scaffolding, and corrective feedback when that is 
required; and it can encourage striving for superior performance when it is linked with social or 
other rewards. 

Program structure conductive to integrity to program models. We expect that the quality 
and strength of implementation of many interventions will depend on the availability of 
structures that promote full and faithful implementation. Such structures include manuals 
specifying the procedures to be used; written implementation standards specifying such things as 
how much, to whom, when, and with what duration interventions are to be applied; and quality 
control mechanisms such as procedures for observing, documenting, or comparing actual 
implementation with. standards for implementation. 

Integration into normal school operations, local initiation, and local planning, (a) Some 
activities or programs are easier to integrate into school activities than are others. Schools are 
characterized by certain pervasive regularities (Sarason, 1971). For example, almost all 
secondary schools hold classes and in most, students move from one time-designated subject- 
matter class to another. Large numbers of people tend to move from class to class at the same 
time, followed by periods of relative quiet with instruction or study occurring. Activities that fit 
into classroom and class periods are easier to integrate into school activities than activities that 
could disrupt the school schedule, (b) Most of the people inhabiting the school are “regulars” - 
that is they are there every day for most of the day. When individuals who are not “regulars” 
enter the school it usually upsets scheduled activities somewhat. For example, when an adult 
must substitute for an absent adult teacher, the class is more disorderly than usual and the orderly 
flow of instruction tends to be disrupted. All of these features of schools are remarkably similar 
from school to school, so much so that a school might seem highly unusual if even one of these 
features were altered. We expect that prevention interventions which are matched to the 
regularities of the typical school will tend to be implemented in stronger form than those which 
either go outside of the regularities of the school or disrupt it. For example, activities which 
disrupt class schedules by pulling students from classes or requiring people to leave the premises 
will be difficult to implement. Activities which involve “regular” inhabitants of the school will 
operate more dependably than those which rely on persons who are occasional inhabitants 
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(visitors), (c) Some parts of school programs are mandated by state or local education agency 
regulations, and other activities or arrangements arise locally through the choices, initiative, or 
habitual ways of acting of school insiders. Programs, activities, or arrangements attempted 
through the intervention of school outsiders often generate resistance. Sometimes this is because 
they are not well matched with the regularities of the school to which a school’s inhabitants are 
accustomed. Sometimes this is because the proposed innovation competes with priorities of 
those in a school. Sometimes it is because of a history of ill will or resentment. Whatever the 
reason, activities developed or selected by school insiders may be easier to implement in schools, 
(d) Quantitative synthesis of previous research (Lipsey, 1992) has suggested that interventions 
implemented by researchers are usually more effective than those implemented by others. 

Several interpretations of this observation are possible. One is that research personnel make use 
of more information, more valid information, or more effective techniques in devising 
interventions. A second interpretation is that research personnel attend more to problems of 
strength and integrity of program implementation. 1 We hypothesize, that the use of information 
in selecting or designing prevention activities is important, and that those schools making use of 
more or better information will implement sounder programs. Information may be provided by 
researchers or experts, technical assistance providers, media, or other sources. 

Feasibility. People arrive at the school pretty much all at once at a designated common 
starting time in the morning and most formal activity ends in the afternoon when most persons 
leave the premises. Few people come and go during the school day except at its beginning and 
end. Activities that take place during the regular school day can be more easily implemented than 
those that take place outside this time interval. Other obstacles also sometimes impede- . > 

feasibility. These may include the requirement for special resources or materials not generally 
available, transportation, and so on. 

Level of disorder. Finally, everything is easier to accomplish in an orderly school. Certainly 
this is true of instruction. School disorder is expected to make the implementation of any 
intervention more difficult to implement, and this includes interventions to prevent or reduce 
disorder. School orderliness is an element of organizational capacity (listed first above), but we 
list it separately because of its special importance in the present context. 

This list of factors linked to implementation level is derived from our efforts to understand 
the success and failure of implementation of programs directed at reducing delinquency in 
schools. It is distilled from the review of factors associated with implementing and sustaining 
innovations (Berman & McLaughlin, 1978; McLaughlin, 1990), evaluations of school-team 
approaches to reducing school crime (Social Action Research Center, 1979, 1980), research on 
the role of leadership in improving educational programs (Hall, 1987; Hall, Hord, Huling, 
Rutherford, & Stiegelbauer, 1983; Hord, 1981); the more general research on leadership (Clark & 
Clark, 1990; Yukl & Van Fleet, 1992); reviews of effective school reforms (Miles, 1980, 1986; 



'Another possibility is that researchers tend to select schools where implementation is easier to 
achieve as research sites. 
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Miles, Farrar, & Neufeld, 1983), a review of organization development efforts in schools (Fullan, 
Miles, & Taylor, 1980); and research on the implementation of instructional programs (Fullan & 
Pomffet, 1977) as well as from our own work (G. Gottfredson, 1982; G. Gottfredson, 

Gottfredson, & Cook, 1983; G. Gottfredson, 1987b; D. Gottfredson, Gottfredson, & Hybl, 1993; 
D. Gottfredson & Gottfredson, 1992) on implementing and evaluating delinquency prevention 
programs and programs to manage student behavior. 

Goals and Objectives of the Project 

Putting effective prevention programs in place requires that the field confront the problem of 
quality of implementation and build effective strategies to enhance it. The first aim in the present 
research, therefore, has been to describe the range of prevention program types being 
implemented in school-based programs and to test the validity of factors hypothesized above as 
affecting implementation. The present study addresses these issues. It also seeks to devise 
workable measures of quality of prevention activity implementation that can be put into 
operation through surveys of program implementers, to provide a description of what is being 
done and how well, and to provide a first thorough account of the nature and extent of what 
schools now do to prevent problem behavior and to promote safe and orderly environments. 

Classification and description of existing programs. The first product of the present effort is 
a classification of school-based prevention activities in terms of rationale (objectives) and 
program model. A taxonomy was needed to allow for estimates of the incidence of each type and 
combination of types. This report provides these estimates. 

Empirical validation (tests) of predictive factors. A second product is a set of empirical 
predictors of success in implementing prevention programs. These research-based indicators, 
which are based in observable features of program design, location, arrangements, staffing, and 
so forth should be useful in (a) selecting promising activities or programs, (b) allocating the 
appropriate level and type of assistance necessary to foster high quality implementation, and (c) 
understanding why certain programs do not produce the expected results. They should find 
additional applications as program assessment tools. 

Program assessment tools. In the course of this research we have constructed instruments to 
assess the level and content of prevention activity as well as brief survey-based assessments of 
school organizations, individuals, program materials, training, structures and arrangements. A 
number of these indicators have been shown to be predictors of quality or extensiveness of 
program implementation and should have use as tools to diagnose program problems and 
pinpoint areas where assistance or development is needed if a program is to be successfully 
implemented. 
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Plan of the Research 



The design for the research called for the collection of five main kinds of information by 

executing an equal number of steps. 

1 . Examples of prevention and intervention models being used in schools were collected, 
examined and classified to develop a comprehensive taxonomy of activities. To locate 
activity types, we scrutinized lists of activities recommended by government agencies, 
technical assistance providers, professional organizations, promotional literature, regional 
educational laboratories, and the scientific literature. The resulting taxonomy guided the 
development of other data collection instruments. This first step was completed at the end 
of 1996. 

2. Principals in a national probability sample of schools were surveyed to identify activities 
their schools had in place to prevent or reduce delinquency, drug use, or other problem 
behavior or to promote a safe and orderly school environment. They indicated if their school 
had activities of various types, named the activities, and provided the names of individuals 
who could provide details about each activity named. The resulting lists of school 
prevention and intervention programs were used to sample prevention activities in a 
subsequent step. Principals also described features of their schools and reported on past 
experiences with the implementation of programs and on school staffing. These surveys 
were conducted in the spring, summer, and early fall of 1997. 

3. Individuals knowledgeable about school prevention activities (called “activity coordinators”) 
were surveyed to obtain detailed descriptions of specific prevention activities and to describe 
certain features of their school. To conduct these surveys, we developed a set of fourteen 
activity coordinator questionnaires corresponding to fourteen categories in our taxonomy of 
prevention models. To the extent possible, the questionnaires for all categories were 
parallel. Thus, although the specific content of questionnaires for different areas was 
appropriate for activities of each type, the nature of information sought was parallel. 
Wherever possible each questionnaire sought information about the extent to which best 
practices were used, about the extensiveness of student exposure, about tr ainin g, and so 
forth. Activity coordinators also reported about themselves and about school support and 
supervision for prevention activities. These surveys were conducted in the spring of 1998. 

4. Teachers and students in participating schools were surveyed to obtain their reports of their 
own participation in prevention activities, about prevention activities in the school, and to 
obtain reports about victimization, safety, delinquent behavior, school orderliness, and other 
aspects of school climate. These surveys were conducted in the spring of 1998. Generally, 
all teachers in participating schools were sampled, and a sufficient number of students were 
sampled to produce an estimated 50 respondents per school. 
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5. Principals were surveyed for a second time in the spring of 1998. They reported about 
school wide disciplinary policies and practices, crimes occurring in the school, certain 
school-wide arrangements such as scheduling, architectural features of the school, and other 
characteristics of the school about which the principal was the most appropriate informant. 
Principals also reported about their own practices, biographical history, and personality style. 

Table 1 . 1 summarizes the surveys conducted and the type of information collected in each. 
The table also shows that certain archival information is also available - drawn from the 
Common Core of Data maintained by the U.S. Department of Education or provided by the 
mailing list vendor. 

The sample was designed to describe schools in the United States and to describe schools by 
level and location. Accordingly a sample of public, private, and Catholic schools, stratified by 
location (urban, suburban, and rural) and level (elementary, middle, and high) was drawn. A 
probability sample of 1287 schools (143 for each cell in the sample design) was selected with the 
expectation that if a response rate of 70% could be achieved there would be 300 schools 
responding at each level and 300 schools responding from each location (about 100 per cell or 
900 schools overall). 

Conducting Surveys and Participation Rates 

Phase 1 Principal Survey. In conducting the phase 1 principal survey (PQ1), we determined 
that of the 1287 entities sampled, 7 were found to be closed and one not to be a school - leaving 
1279 schools in the sample. In addition, the location or level classifications were found to be 
incorrect for some schools, so the number of actually sampled schools is sometimes greater and 
sometimes less than 143 per cell. 2 Overall, useful responses were received from 848 schools in 
PQ1, 66.3% of those from which responses were sought. Table 1.2 shows that the participation 
rates ranged from a low of 59.0% among urban high schools to a high of 74.6% among rural 
elementary schools. 

The effort that was required to obtain completed questionnaires from schools far exceeded 
our expectations. One indication of the difficulty involved are the counts of telephone contacts 
with schools that were required to obtain cooperation. In all, we completed 8,783 telephone calls 
to schools to request PQ1 data. The number of calls per school ranged from 0 (some schools 
returned questionnaires without having to be called) to 36. The average number of telephone 
calls made to schools that had to be called at least once was 7.9 completed calls. In addition, 
survey materials were resent once by Federal Express to 964 schools that had not responded. 



2 The location codes obtained from the mailing list vendor (the original source of which was the 
Common Core of Data developed by the U.S. Department of Education) were often in error. It 
appears that many schools were misclassified as to location in the CCD. Efforts were made to 
identify and reclassify misclassified schools. 
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Table 1.1 



Measures Employed in the National Study of Delinquency Prevention in Schools 









Source of Information 








Principal 






Activity 

coordinators 


What is measured 


Archives 


PQl 


PQ2 


Teachers 


Students 


Grade levels 










/ 




Demographic characteristics 




/ 




/ 


/ 




School safety 








/ 


/ 




Victimization 








/ 


/ 




Drug use, violence, other 
delinquent behavior, crime 






/ 




/ 

t 




School climate — morale, 
discipline related, 
organizational capacity 






/ 


/ 


/ 


/ 


Level of implementation of or 
exposure to prevention 
activities 






/ 


/ 


/ 


/ 


Correlates of problem behavior 
Leadership style of principal 






/ 


/ 


/ 




Personality 






/ 






/ 


Biographical information 






/ 






/ 


Organizational origins of 
activities 






/ 






/ 


Funding sources 






/ 






/ 


Nature & extent of training 






/ 






/ 


Program/activity features 






/ 






/ 


Staff stability vs. turnover 












/ 


Relation of implementers to 
developers 






/ 






/ 



Note. PQ1 = Principal Questionnaire for Program Identification; PQ2 = phase 2 Principal 
Questionnaire. Certain additional information collected by Westat for a small number of schools 
that were site-visited and in a survey of local education agency personnel is not covered by the 
present report. 
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Table 1.2 

Principal Phase 1 Questionnaire Response Percentages by Level and Location 

School Level 
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Many schools still indicated to our callers that they had not received or had misplaced the 
questionnaires, and our response was to mail another set of replacements. This was done for 53 1 
schools (42% of the sample). Replacements for “lost” questionnaires were resent twice to 1 18 
schools (9%) and three times to 21 schools (2%). When we could obtain school telefax numbers, 
we sent faxes to nonresponding schools. One telefaxed request was sent to 225 schools and two 
telefaxed requests were sent to 1 3 schools. Information about effort required to obtain data in 
phase 1 is summarized in Table 1.3. 

Cooperation from schools was more difficult to obtain than we anticipated. We noted a 
reluctance to cooperate with surveys on the part of many principals, who often see themselves as 
overburdened with surveys and are cynical about their value. Some districts have erected barriers 
to research. Evidently a growing number of district data collection requirements, educational 
dissertation research projects, and requirements that programs be evaluated has lead to greater 
resistance to research in recent years — although few of these evaluations or dissertations 
contribute to the literature. We speculate that the large number of evaluations — often required by 
funding agencies - that contribute little or nothing to knowledge because they are not even 
published, actually become an obstacle to the development of knowledge because they make 
serious research more difficult to conduct. 

Collaboration by Westat in Phase 2 Data Collection. In view of the difficulty in obtaining 
data from schools, we sought ways to bring greater resources to the research. NIJ personnel 
assisted us in working with the Department of Education to bring about a merger of our ongoing 
study and resources intended to address similar problems in the form of a contract ED had with 
Westat, Inc., to gather information about school violence and programs sponsored by the Safe 
and Drug Free Schools and Communities Act. We proposed that Westat assist in collecting 
student and teacher surveys in a sample of about 600 secondary schools, and that the data 
collected by GAInc and Westat be shared. A memorandum of understanding was agreed upon by 
ED and NIJ to formalize the cooperative enterprise. 

One implication of the involvement of Westat (under contract from ED) was that the teacher 
and student survey instruments had to go through an approval process at the ED and then the 
OMB approval process. That process was completed just in time to collect data in the spring of 
the 1 997-98 school year. A second implication was that a somewhat revised approach to 
recruiting secondary schools to participate in the second phase of surveys was taken. In keeping 
with traditional ED and Westat approaches, first State Departments of Education and then local 
education agencies were approached to seek their concurrence with the surveys to be conducted 
in the second phase (another principal questionnaire, program implementer questionnaires, 
student questionnaires, teacher questionnaires, and - to answer questions important to ED — local 
education agency questionnaires). This change in strategy was not adopted for phase two surveys 
of elementary schools, except for those elementary schools in school districts in which Westat 
would be seeking the participation of secondary schools. 
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Table 1.3 

Steps Taken to Obtain Responses in Phase 1 Principal Survey 



Initially Planned Implemented 

Heads up mailing to 1287 principals indicating that 
questionnaire will be coming 

Initial mailing of 1287 questionnaires 
1213 reminder post-cards 

1112 schools required calls because they did not return 
materials without one; 8,783 completed phone calls; 7.9 
telephone contacts per school that initially failed to respond 
(range 1-36) 

38 requests for district approval for principal to answer 
questions filed 

964 second survey deliveries by Fed Express with 
questionnaires, personal note, letters from National 
Association of Elementary School Principals and National 
Association of Secondary School Principals 

Replacement mailings of 670 additional replacement delivery to principals who lost 
survey materials or discarded materials (531 once, 1 18 twice, 21 three times) 

6 principals interviewed 

±751 mailings with letter from NIJ director 

238 telefax requests for completion 



Westat personnel identified “on-site coordinators” in secondary schools and GAInc 
personnel identified “on-site coordinators” in elementary schools. These individuals assisted in 
the collection of the surveys conducted in the second phase. 

Because a primary purpose of the research is to learn about the implementation of prevention 
activities, participation in phase 2 logically depends on participation in phase 1 . Nevertheless, 
because excluding those schools that did not participate in phase 1 from the collection of data 
about school safety, victimization, and other characteristics of principals, teachers, students, and 
schools would necessarily limit the representativeness of those data, we decided to make an 
effort to include in the phase 2 data collection effort every school that had not affirmatively 
refused in phase 1 . A short form prevention activity screener was used to identify over the 
telephone or by telefax prevention activities for schools that had failed to provide this 



Initial mailing of 900 
questionnaires 



Telephone contact with 
school to seek return 
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information in phase 1. Teacher and student questionnaires were sought only in secondary 
schools. Principal and prevention activity questionnaires were sought in all schools in the 
sample (unless they had affirmatively refused to participate in phase 1). 

As expected given the extensive effort to secure participation in phase 1, we were not highly 
successful in obtaining cooperation from schools that had failed to participate in that first phase. 
Accordingly, final response rates may be viewed essentially as the product of phase 1 and phase 
2 response rates. For example, if we obtained a 70% response rate in phase 1 and a 70% 
response rate in phase 2 we would expect to obtain a 49% response rate (.7 x .7 = .49). For some 
categories of schools we exceeded this expectation, and for some categories we did not. 

Phase 2 Principal Survey. In conducting the phase 2 surveys, an additional school was 
found to have been closed, leaving 1278 schools in the sample. Table 1.4 shows the phase 2 
response rates and number of respondents to the phase 2 principal questionnaire. Again, 
obtaining cooperation was most difficult in urban schools, where completed phase 2 principal 
questionnaires were obtained for 45.5% of the sample. Rural schools were more cooperative, 
and we obtained completed phase 2 principal questionnaires from 57.1% of rural schools. 
Participation ranged from a low of 39.6% for urban high schools to 58.4% for rural middle 
grades schools. 

Student Survey. We sought the completion of student questionnaires in all secondary 
schools. Westat personnel obtained rosters of students, and students were systematically 
sampled (separately by sex or grade level where it was possible to obtain that information) using, 
a sampling fraction that was expected to yield 60 student respondents per school. Usable 
questionnaires were completed by 16,014 students. Table 1.5 shows information about 
participation rates for schools in the student survey. Schools with poor levels of student 
participation are treated as nonparticipants in Table 1.5. Overall, 36.4% of the secondary schools 
from whom participation was sought in student surveys participated at a useful level. As before, 
the table shows that participation was better in rural schools than in urban schools, and it was 
better in middle/junior high schools than in high/vocational/combined schools. Participation 
ranged from a low of 22.8% of urban high schools to 50.4% of rural middle/junior high schools. 

Teacher Survey. We sought the completion of teacher questionnaires in all secondary 
schools, and usable questionnaires were completed by 13,103 teachers. Table 1.6 details the 
participation rates. Again, the table shows that rural schools were much more cooperative than 
suburban or urban schools. Participation ranged from a low of 39.0% of urban high schools to 
59.1% of rural middle/junior high schools. 

Although principals of Catholic schools participated in the phase 1 and phase 2 principal 
surveys at rates comparable to public school principals, few Catholic and private schools 
cooperated with efforts to include students and teachers in surveys, as Table 1.7 shows. Only 9 
of the 3 1 Catholic secondary schools in the sample participated in student surveys, and only 1 7 of 
105 private schools participated. The low rates of participation by Catholic and private schools, 
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Table 1.4 

Princip al Phase 2 Questionnaire Response Percentages by Level and Location^ 

School level 
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school level. Of the 1287 entities, eight were found to be closed and one not to be a school - leaving 1278 schools in the sample. In 
addition, the location or level classifications were found to be incorrect for some schools, so the number of actually sampled schools 
sometimes greater and sometimes less than 143. 



Table 1.5 



Phase 2 Student^ Questionnaire School Participation Percentage s by Level and Location 



Metropolitan status 


Middle or 
junior high 


School level 

High, 

vocational, 

combined 


Total 


Rural/non-metro 


N participating 


69 


56 


125 


% participating 


50.4 


38.6 


44.3 


N sampled 


137 


145 


282 


Suburban 


N participating 


57 


41 


98 


% participating 


38.0 


29.9 


34.1 


N sampled 


150 


137 


288 


Urban 


N participating 


56 


31 


87 


% participating 


38.9 


23.1 


31.3 


N sampled 


144 


134 


278 


Total 


A participating 


182 


128 


310 


% participating 


42.2 


30.8 


36.6 


N sampled 


431 


416 


847 



Note. Classification of schools by level and location reflects school status known to research 
team at time of phase 2 survey. Participation is defined as completion of a usable number of 
student questionnaires. (See Appendix B.) 



combined with the relatively small number of such schools in the representative sample, implies 
that it will seldom be appropriate to examine separate estimates for these schools. 

Activity Coordinator Survey. Activity questionnaires were used to obtain detailed 
descriptions of the nature, level, and quality of implementation of specific prevention activities. 
These activity questionnaires also sought additional information about the school. From the total 
sample of prevention activities identified in phase 1, we sampled one activity in each of 14 
categories per school. In addition, we sampled all D. A.R.E. and peer mediation programs 
because of special interest in these particularly popular prevention programs. This sampling 
could result in up to 16 activities sampled per school. Sometimes, the principal had named the 
same individual as knowledgeable about two or more of the activities that turned up in our 
sample. When this occurred, we made an effort to determine in a telephone inquiry of the 
school’s principal whether each activity still existed in the school and to get the principal to 
identify different individuals capable of describing each of the sampled activities. If we were 
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Table 1.6 



Phase 2 Teacher^ Questionnaire^ School Particijaation Percentages^ by Level_ and Location 







School level 






Middle or 


High, vocational, 




Metropolitan status 


junior high 


combined 


Total 


Rural/non-urban 








N responding 


81 


75 


156 


% responding 


59.1 


51.7 


55.3 


N sampled 


137 


145 


282 


Suburban 








N responding 


70 


54 


124 


% responding 


46.7 


39.4 


43.2 


N sampled 


150 


137 


287 


Urban 








N responding 


70 


53 


123 


% responding 


48.6 


39.6 


42.4 


N sampled 


144 


134 


278 


Total 








N responding 


221 


182 


403 


% responding 


51.3 


43.8 


47.6 


N sampled 


431 


416 


847 


Note. Classification of schools by level and location reflects school status known to research 


team at time of phase 2 survey. Participation is defined as completion of a usable number of 


teacher questionnaires. 








Table 1.7 








Survey Participation Rates by School Auspices 










Auspices 




Survey 


Public (N = 1041) 


Catholic (N= 88) 


Private (N= 149) 


Principal Phase 1 








n responding 


696 


63 


89 


% responding 


66.9 


71.6 


59.7 


Principal Phase 2 








n responding 


' 537 


47 


51 


% responding 


51.6 


53.4 


34.2 


Student 








n secondary schools 


711 


31 


105 


n responding 


284 


9 


17 


% responding 


39.9 


29.0 


16.2 


Teacher 








n secondary schools 


711 


31 


105 


n responding 


359 


15 


29 


% responding 


50.5 


48.4 


27.6 
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Table 1.8 



Program Coordinator Survey Response Rate 





N 


% of all 


%of 

requested 


Initially sampled activities 


8043 


100.0 


— 


Determined to exist a 


5067 


63.0 


— 


Determined not to exist, de-selected 3 


796 


9.9 


— 


Existence undetermined 3 


2180 


27.1 


— 


Activities remaining in sample at survey time 


7247 


90.1 


' 


De-selected b 


127 


1.6 


— 


Sent incorrect booklet 0 


16 


0.2 


— 


Potential responses 


7104 


88.3 


100.0 


Responded 


3691 


45.9 


51.9 


Refused 


668 


8.3. 


9.4 


Other non response 


2745 


34.1 


38.6 



3 At time of phase 2 pre-survey telephone inquiry 

b De-selected to avoid overburdening individual respondents. Each individual was limited to 
describing two activities. 

c Program Coordinator Questionnaire for the wrong activity type sent through researcher error. 



unsuccessful in this attempt to “unburden” respondents by obtaining substitute respondents, we 
re-sampled so that a person was not asked to describe more than two activities. A summary of 
the result of effort to obtain completed Activity Coordinator questionnaires is presented in Table 
1 .8. Of 8,043 initially sampled activities, we sent booklets for 7,104 activities to identified 
individuals. Of these, 3,691 were completed (45.9% of all sampled activities and 5 1 .9% of the 
activities for which completion was requested). 



Level of Effort Required to Collect Phase 2 Data. Here we provide information about the 
level of effort required to collect phase 2 data by summarizing the amount of contact with 
schools required to obtain the principal and activity questionnaires from elementary schools. 



Of the 432 elementary schools in the initial sample, 102 had affirmatively refused to 
participate. We made no further contact with these schools. Of the 330 remaining schools, 20 
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never refused and also never provided any information or otherwise participated and we did not 
initiate phase 2 activity. This left 310 elementary schools at which we directed effort to collect 
phase 2 data. These 3 1 0 schools were contacted by telephone or telefax a total of 2,993 times. 
These telephone contacts were frustrating because the most common outcome, occurring for 69% 
of the calls, was the requirement that we call back again at another time. The mean number of 
calls per school in phase 2 was 9.7. The range in the number of contacts per school was 1 
(refusing schools) to 25 (difficult cases). A quarter of the schools required more than 13 phase 2 
contacts. 

To reduce the problem that we had observed for phase 1 of schools indicating that they had 
not received or had misplaced survey materials, we did not use the U.S. postal service for 
delivery or return of survey materials. Instead, we used a service provided by United Parcel 
Service that allowed us to track the status of each item and tell school personnel the name of the 
adult who had signed when the item was received. Although this service was expensive, it very 
much reduced the problem of misplaced survey materials. Use of this procedure required a 
minim um of two additional contacts with UPS for each school (out and return) plus more 
contacts whenever there was a delay in delivery or return, or when a school claimed that a 
package was not received. 

Information has not been tabulated for the secondary schools for which Westat handled the 
data collection. That effort was funded at a higher level, involved a larger staff, and made use of 
Federal Express rather than UPS delivery. It was complicated by the requirement of obtaining 
concurrence of the districts in which schools were located, and was made more difficult by the 
additional burden of teacher and student surveys (see Crosse, Burr, Cantor, & Hantman, 2000). 

Reasons for Nonparticipation. Additional exploration of patterns of nonparticipation was 
made by examining information about the location of schools in the sample from the 1 990 census 
of population using school zip code to geocode the schools. Details of the correlations of zip 
code level community characteristics with survey participation are provided in Appendix Table 
Bl.l. Urbanicity was the most robust correlate of participation. We also tried to understand 
refusal by tabulating the reasons given by those who affirmatively refused to participate in the 
phase 1 principal survey. In some cases, a policy of not participating in surveys was cited. Most 
often, however, principals indicated they were too busy or the burden imposed was too great. 
Details are shown in Appendix Table B1.2. Additional insight into school and district 
nonparticipation is provided by Crosse et al. (2000). 

Organization of the Remainder of the Report 

Chapter 2 describes the nature and extent of problem behavior in schools. It organizes 
reports by principals, teachers, and students, about crime and problem behavior. It also presents 
information about student and teacher perceptions about the safety of their schools. Comparisons 
with some other sources of information about problem behavior in schools are made. 
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Chapter 3 describes activities in schools to prevent or reduce problem behavior or to 
promote a safe and orderly environment. It begins by describing the development of a 
comprehensive classification of prevention activities, programs, and arrangements in schools. 
Then empirical evidence on the extent of deployment of these activities is summarized. It 
describes school wide policies and arrangements, school rules and discipline practices, and the 
nature and extent of discretionary activities to prevent problem behavior or promote safety in 
schools. 

Information about program intensity and the extent to which school activities employ “best 
practices” is summarized in Chapter 4. This chapter explains the importance of program 
intensity and fidelity to good practices. It describes the measures of intensity and fidelity to good 
practices employed in the present research, provides a structure for assessing the adequacy of 
school prevention activity, and describes the variability observed in program quality. 

Chapters 5 and 6 summarize evidence about the correlates of program quality - testing some 
hypotheses about the conditions and arrangements that make quality program implementation 
possible. 

Finally, Chapter 7 offers recommendations based on information developed in this inquiry, 
and it offers speculations about potentially useful practices. 

Appendices contain details about measurement and methods that are not necessarily 
described in the body of the report. For example, information about the content of scales used to 
measure constructs involved in the research and about their measurement properties is provided 
in appendices. 
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The Nature of Problem Behavior in Schools 

In this chapter we describe the amount of problem behavior that occurs in schools, what 
form it takes, and how it is distributed. We first describe the amount of crime according to 
principal reports. Second, the nature, amount, and distribution of classroom disorder and 
personal victimization according to teacher reports is described. Third, student reports are 
examined for an account of student delinquent behavior, drug use, and personal victimization. 
Then the reports of both teachers and students of their perceptions of school safety are described. 
Finally, the information developed in the National Study of Delinquency Prevention in Schools 
(NSDPS) is compared with information developed in different ways, and we call attention to the 
variability in school safety that produces relatively higher levels of delinquency and disorder in 
some schools serving urban middle school aged youths. 

How Much Problem Behavior Occurs? 

In nationally representative surveys, the NSDPS estimated the amount of problem behavior 
that occurs in schools by asking principals, teachers, and students to report on problem behavior, 
victimization, and school disorder. 

Principal Reports 

One way of estimating the amount of delinquent behavior occurring in schools is to ask 
principals about it. In the NSDPS survey of principals in the spring of 1998 we asked 
respondents to tell us how many crimes of various types had been reported to law enforcement 
representatives during the 1997-98 school year. The percentages of schools reporting at least one 
incident for each of five crime categories are displayed in Table 2.1. Nationwide, 6.7% of 
schools or an estimated 6,451 schools reported at least one incident of physical attack or fight 
with weapon to law enforcement personnel during the year. Some schools reported more than 
one such incident, so an estimated 20,285 fights or attacks with a weapon were reported to 
authorities according to our survey. 1 

A small percentage (2.2%) of elementary schools reported fights or attacks involving a 
weapon, for an estimated 2,801 such incidents in elementary schools. Fights or attacks with 
weapons are most common in middle schools - 21.0% of middle/junior high schools reported 



'Table 2.1 provides a summary of the more detailed information tabulated in Appendix H. The 
appendix tables provide estimated numbers of incidents and numbers of schools with incidents. 
Unless otherwise noted, tables in the main body of this report are adjusted for non-response and 
weighted to represent all schools, teachers, principals, or students in the nation. Standard errors 
or confidence intervals presented are calculated using a re-sampling method (the jackknife) to 
account for the complex sample design employed. 
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Table 2.1 

Percentage of Schools In Which One or More Incidents of Crime Was Reported to Law Enforcement - 1997-98 School Year 
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these incidents, for an estimated 7,576 incidents. The percentage of high schools 2 reporting a 
physical attack or fight involving a weapon (1 0.6%) is lower than the percentage for middle 
schools, but there were more such incidents per school reporting at least one incident so that the 
estimated number of fights or attacks with a weapon reported is 9,909. The percentages of 
schools reporting a fight or attack with a weapon do not differ significantly by location. 

Robbery shows a similar pattern, with 5.9% of all schools reporting at least one robbery. A much 
higher percentage of middle schools reported at least one robbery than did elementary schools. A 
higher percentage of high schools than elementary schools reported at least one robbery (the 
percentages of middle and high schools reporting at least one robbery are not significantly 
different). A smaller percentage of rural schools than other schools reported robberies. 

Physical attacks without a weapon, theft or larceny, and vandalism are much more common in 
schools than are the more serious incidents. Forty-four percent to 49% of all schools reported 
crimes of these types to the authorities. The percentages were again highest for middle schools, 
although the percentages of middle and high schools reporting at least one incident of vandalism 
to the police were about the same. Because 72% of middle schools reported at least one attack or 
fight without a weapon, it is fair to say that some fighting is typical of middle schools. 

The percentages of nonpublic (Catholic or other private) schools in which at least one incident 
was reported to law enforcement personnel are lower than the percentages of public schools for 
each of the five crimes examined. Private and Catholic schools tend to be smaller than public 
schools. 3 The percentages reported in Table 2.1 do not standardize rates for population size. 

An alternative way to describe the distribution of school crime in schools at different levels and 
locations is to form a composite measure that combines reports about all of the crimes about 
which we inquired. Table 2.2 shows such results for a scale composed of principal reports of the 



2 High schools include all schools serving the highest grade levels. Some of these are 
comprehensive schools serving students in grades K-12. Others are vocational schools. More 
details of the sample descriptions are provided in Appendix A. 

3 Based on principal reports in PQ1 enrollments are as follows: Public M= 572, Mdn = 500, 
range = 6 - 4482; Private M= 186, Mdn = 115, range = 4 - 1780; Catholic M= 383, Mdn = 297, 
range = 100 - 1310. 
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Table 2.2 

Means and Standard Deviations for School Crime and Gang Problem Scales from the Phase 2 Principal Questionnaire by School Level and Location 

Elementary Middle/Junior High Total 

Location Value 95% Cl Value 95% Cl Value 95% Cl Value 95% Cl 
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number of incidents reported of each type. 4 The scales are displayed in a T-score metric, where 
the mean is 50 for the nation’s schools and the standard deviation is 10. When displayed in this 
way, it is apparent that the mean crime score for urban middle/junior high schools is over a 
standard deviation above the mean for all schools (T-score = 62.4). Furthermore, the table shows 
that the standard deviation of T-scores for urban middle schools is very large (20.2) compared to 
the standard deviation for all schools. This implies that some urban middle schools report a great 
deal of crime to the police and that there is great variability in the scores for urban middle 
schools. The relatively high crime scores for urban middle/junior high schools is not due only to 
a few extremely high scoring schools. The second panel in Table 2.2 shows that when T-scores 
are trimmed so that no score is allowed to be above 80, the mean for urban middle/junior high 
schools is still almost a standard deviation above the mean for all schools. 

Principals were asked about gang problems in the school and community, and T-scores for a 
scale composed of their responses to two questions about gangs is shown in the bottom panel of 
Table 2.2. Urban principals report more gang problems than do suburban or rural schools, and 
suburban schools report more problems than do rural schools (note that the confidence intervals 
do not overlap). 

We are circumspect about placing too much credence in the principal reports of school crime 
for four reasons. First, principals naturally want to present their schools in a good light and it is 
only to be expected that many principals will be reluctant to notify the police when a crime - 
particularly one that they may regard as minor - occurs in their school because of the negative 
image of the school that this may promote. According to the National Crime Victimization 
Survey (Whitaker. & .Bastian, 1991), only 9% of violent crimes against teenagers occurring in . 
school were reported to the police compared with 37% of such crimes occurring on the streets. 
This same reluctance may influence their reports in a survey. Second, in our experience working 
in schools over the past decades, we have observed that some schools report only a small fraction 
of incidents involving fights or attacks, alarm pulls, thefts, and vandalism to the police. We are 
confident, therefore, that in a non-trivial proportion of schools, many or most categories of crime 
are under-reported. Third, the principal reports show only modest convergence with other 
measures of school disorder in the present research (see Appendix Table G-l) and in prior 
research (G. Gottfredson & Gottfredson, 1985). Fourth, principal reports are the reports of a 
single individual so that individual differences in reporting tendency are confounded with the 
measurement of crime and error is expected to be greater than if there were several persons 
reporting about the school. Accordingly, the reports of teachers, reported next, and of students 
are of interest. 



“The number of crimes of each type is log transformed and standardized (with respect to item 
variances) before being combined to form a scale. Results for untrimmed scores are shown in the 
top panel of Table 2.2, and results for scores that are trimmed to ± 3 standard deviations from the 
mean are shown in the second panel of the table. Scores are not standardized with respect to 
enrollment size. Elementary schools tend to enroll fewer students than secondary schools; and 
enrollments tend to be higher in urban and suburban schools than in rural schools. 
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Teacher Reports 



In secondary schools, teachers were asked to report about their own experiences of 
victimization in the school, about their views on the safety of the school, and about classroom 
disorder. 

The percentages of teachers reporting each of several kinds of victimization in school are 
shown in Table 2.3. Many teachers - 42% overall - report having received obscene remarks and 
gestures from a student; 28% experienced damage to personal property worth less than $10; 24% 
had property worth less than $10 stolen; 21% were threatened by a student; 14% experienced 
damage to personal property worth more than $10; 13% had property worth more than $10 
stolen; 3% were physically attacked. Less than 1% of teachers reported having been physically 
attacked and having to see a doctor or having had a weapon pulled on them. 

Victimization rates are higher in middle schools than in high schools for obscene remarks 
and gestures, minor property damage, minor theft, threats, minor physical attacks, and physical 
attacks requiring physician attention. For all secondary schools, the urban victimization rates are 
higher than the rural rates for threats, serious attack, minor theft, minor attack, major theft, 
obscene remarks, and major property damage. The urban rates are higher than the suburban rates 
for serious attack, minor theft, major theft, minor property damage, minor attack, major property 
damage, threats and obscene remarks. Estimates of the numbers of teacher victimized are found 
in Appendix Tables H2.6 through H2.10. Because so many teachers work in the nation’s 
schools, even small percentages translate into a large number of teachers victimized. For 
.example, although we estimate that 7.9 per 1000 teachers was attacked and had to visit a doctor, 
the number of teachers estimated to have been so victimized is about 12,100 in the 1997-98 
school year. 

Secondary school teachers were also asked to report about classroom disorder and the 
conduct of students in their schools. Table 2.4 shows that 27% of teachers report that student 
behavior keeps them from teaching a fair amount or a great deal. Misconduct that interferes with 
teaching is more common in middle schools than in high schools, and it is more common in 
urban schools than in suburban or rural schools. Reports of other forms of student misconduct 
are shown in Appendix Table H2.1 1. 

Student Reports 

In participating secondary schools, students were asked to report about their own 
participation in a variety of kinds of delinquent behavior and drug use. Interpersonal violence is 
common in middle schools. Table 2.5 shows that 32% of high school students and 41% of 
middle school students reported having hit or threatened to hit other students in the past year. 
Damaging or destroying school property is also relatively common, with about 1 6% of students 
reporting having engaged in this behavior. Whereas middle school students reported 
interpersonal violence more often than high school students, this pattern was reversed for going 
to school when drunk or high on drugs: 9% of middle school students and 17% of high school 
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Table 2.3 

Percentage of Teachers Reporting Personal Victimization This Year in School, by School Level and Location 

Middle/Junior b High c Total d 

Type of victimization 

and location % 95% Cl n % 95% Cl n % 95% Cl 
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Table 2.4 



Percentage of Teachers Reporting That the Behavior of Some Students in Their Classroom 
(T< alking, Fighting, etc.) Keeps Them from Teaching a Fair Amount or a Great Deal, by School 
Category 





Percentage 


95% Cl 


n 


All schools a> b 


27 


25.7 - 29.1 


13197 


Level 








Middle/Junior 


34 


31.5 -36.5 


7351 


High 


24 


21.5 -25.9 


5846 


Location 








Rural 


25 


22.4 - 27.5 


3848 


Suburban 


27 


23.8 -30.6 


4597 


Urban 


31 


28.1 -34.5 


4752 



Note. Percentage = weighted percentage, n - unweighted n. 95% Cl = 95% confidence interval. 



a Percentage differs significantly (p < .001) for school level. 
b Percentage for urban schools differs significantly from rural schools (p < .01). 



students reported having done so. Only 9% of students report having engaged in theft, and about 
5% having hit or threatened to hit a teacher. 

Students were also asked to report on their experiences of personal victimization, and these 
reports are summarized in Table 2.6. The most common form of victimization experienced by 
students according to these reports is minor theft (of items worth less than $1), with 47% of 
students reporting such theft in the present school year. A larger percentage of middle school 
students (54%) than of high school students (44%) reported experiencing a minor theft. 
Victimization by theft of items worth more than $1 was also reported by a higher percentage of 
middle school students (49%) than of high school students (42%). 

Almost one in five students reported being threatened with a beating, and again this was a 
more common experience for middle school students (22%) than for high school students (16%). 
Victimization by physical attack was reported by 19% of middle school students and 10% of high 
school students. Having things taken by force or threat of force was also more common for 
middle school students than high school students. About 5% of secondary students report having 
been threatened with a knife or gun. Percentages of students reporting theft or attack in the last 
month are roughly half the percentages reporting theft or attack this year in school (see Appendix 
Table H2. 12). 
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Table 2:5 

Percentage of Students Reporting Personal Participation in School Delinquency and Drug Use in Past Year , by School Level and Location 

Middle/Junior High Total 

Self-reported behavior and location % 95% Cl n % 95% Cl n % 95% Cl 

Purposely damaged or destroyed property 
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How Safe Are Secondary Schools? 

V In addition to asking principals about specific instances of crimes reported to law 
enforcement representatives and about teachers’ and students’ personal experiences, 
questionnaires asked secondary school teachers and students about their perceptions of school 
safety and about exposure to violence. This section summarizes their reports about safety and 
exposure to violence. It is important that schools not only be safe, but that people feel safe and 
not fearful in schools. 

Teacher Perceptions 

Secondary teachers usually reported that most places in their schools were fairly safe, 
although perceptions of safety differed according to specific location, as Table 2.7 shows. 
Teachers generally rate their classroom while teaching as safe (a rating of 3.4 on a 4-point scale 
where 3 = fairly safe and 4 = very safe). Other places in the school are generally seen as less safe 
than classrooms during instruction. Locker room or gym and restrooms used by students 
received the lowest ratings for safety (both at 2.7 of the 4-point scale, where 2 = average and 3 = 
fairly safe). Appendix Table H2.13 provides details of the perceived safety of specific locations 
within schools by school level and location. 

An alternative way to describe the distribution of orderliness, victimization, and safety in 
schools at different levels and locations is to form composite scales that combine reports for 
multiple items. Results for such scales are displayed in a T-score metric - where the mean is 50 
for the nation’s schools and the standard deviation is 10 - are shown in Appendix Table H2.14. 
The mean score for classroom orderliness for urban middle/junior high schools is a standard 
deviation below the mean for all schools (T-score = 40.0). The mean score for victimization for 
urban middle schools is over four fifths of a standard deviation above the mean for all school (T- 
score = 58.5); and the mean score for safety for urban middle schools is also somewhat low (T- 
score = 44.3). Middle schools are seen to be less orderly and to be characterized by more 
victimization than are high schools according to the results shown in Table H2.14. 

Student Perceptions and Exposure to Violence 

One way of ascertaining whether students feel safe in school is to ask them if there are 
specific places that they avoid because someone might hurt or bother them there. The 
percentages of students who would avoid each of seven locations in their schools and two 
locations in their neighborhoods are shown in Table 2.8. About 1 1% of students say they would 
avoid certain places on school grounds, and 1 1% say they would avoid school restrooms. In 
general, about a tenth of students say they would avoid the places in school we asked about. 
About a tenth of students also say that they would avoid being outside on the street where they 
live. A larger percentage (16%) would avoid some other place in their neighborhood. 

There are often large differences in perceptions of safety for students of different race/ethnic 
groups. Students who identified a racial/ethnic identity other than White tend to avoid more 
places in school and their neighborhood than do White students. Details are shown in Appendix 
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Table 2.7 

Mean Teacher Reports of Safety from Vandalism, Personal Attacks, and Thefts, in Specific 
School Locations 



Location 


Mean 


95% Cl 


N 


Your classroom while teaching 


3.4 


3.41-3.48 


13038 


The cafeteria 


3.0 


2.97 - 3.07 


12571 


Empty classrooms 


3.0 


2.96 - 3.05 


12665 


Hallways and stairs 


2.9 


2.87-2.97 


12894 


Parking lot 


2.8 


2.80-2.91 


12842 


Elsewhere outside on school grounds 


2.8 


2.78-2.88 


12851 


Locker room or gym 


2.7 


2.65-2.76 


11420 


The restrooms used by students 


2.7 


2.61-2.74 


12807 



Note. Mean = weighted mean. N= unweighted number of respondents. 95% Cl = 95% 
confidence interval. Teachers rated the safety of places on the following scale: 0 = very unsafe, 
1 = fairly unsafe, 2 = average, 3 = fairly safe, 4 = very safe. 



Table H2.15. For example, 15% of Black students, 1 1% of Asian or Pacific Islander students, 

9% of American Indian or Alaskan Native Students, 1 1% of other non-Hispanic students, and 
11% of Latino students 5 say they would avoid certain entrances into the school, but only 6% of 
White students indicate that they would avoid an entrance. Although the number of students 
identifying themselves as Asian or Pacific Islanders or as American Indians or Alaskan Natives is 
relatively small, the reported tendency to avoid certain places is sometimes statistically 
significantly higher than the tendency reported by Whites. Racial/ethnic minority students also 
tend to report avoiding places in their neighborhoods more often than do White students. 

Middle school students avoid places in school because someone might hurt or bother them 
considerably in considerably higher percentage than do high school students. For example 11% 
of middle school students versus 7% of high school students avoid an entrance into the school, 

1 1% of middle school versus 7% of high school students avoid parts of the school cafeteria, 14% 
of middle school and 9% of high school students avoid school restrooms. Middle school 
students also report avoiding places on the street where they live and elsewhere in their 
neighborhoods in higher percentages than do high school students. (See Appendix Table H2.15.) 



5 A pair of racial/ethnic self-identification questions that have been used on some past 
government data collection efforts was used. The first of these questions uses the categories 
White, Black, Asian or Pacific Islander, American Indian or Alaskan Native, and Other. The 
second of these asks for information on Spanish or Hispanic origin. As a result, a large fraction 
of respondents select the “Other” response to the first question. Many of these individuals 
indicate that they are of Spanish or Hispanic origin. Persons of Spanish or Hispanic origin may 
belong to any of the race/ethnic categories. 
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Table 2.8 



Percentage of Students Who Report Staying Away From Specific Places Because Someone 
Might Hurt or Bother Them There 



Place 


Percentage 




95% Cl 


N 


Places in school or on the way to school 


Other places on the school grounds 


11 


10.4 


- 12.6 


15965 


Any school restrooms 


11 


10.2 


- 12.1 


15964 


Any hallways or stairs in the school 


10 


8.8 


- 10.6 


15974 


Other places inside school building 


10 


8.6 


- 10.4 


15964 


The shortest way to school or the bus 


10 


8.8 


- 10.9 


15946 


Parts of the school cafeteria 


9 


7.8 


- 9.4 


15978 


Any entrances into the school 


8 


7.4 


- 9.4 


15977 


Places away from the school 


Any other place in your neighborhood 


16 


14.9 


- 18.2 


15970 


Outside on the street where you live 


10 


9.1 


- 11.1 


15977 



Note. Percentage = weighted percentage. 95% Cl = 95% confidence interval for percentage. 
N = unweighted number of respondents. 



Urban students avoid places in the school and in their neighborhoods in higher percentages than 
do rural students. 



In one set of questions, we asked students about their exposure to violence “this year in 
school.” Responses are summarized in Table 2.9, which shows that 28% report having seen a 
teacher threatened by a student, 20% report having had to fight to protect themselves, and 1 2% 
report having seen a teacher hit or attacked by a student. As with perceptions of safety, there are 
differences in exposure to violence according to race/ethnicity. Among students who identify 
themselves as Black, 40% report having seen a teacher threatened by a student. This is higher 
than the 27% of White students who report having seen a teacher threatened by a student. A 
smaller percentage (18%) of students who identify themselves as Asian or Pacific Islanders 
report having seen a teacher threatened by a student. For two of three questions about exposure 
to violence, boys report more exposure than do girls and middle school students report more 
exposure than do high school students. The difference is particularly large for reports that the 
student “had to fight to protect yourself,” with 28% of boys and 12% of girls answering in the 
affirmative. Details are presented in Appendix Table H2. 1 6. 



An alternative way to describe the distribution of victimization and safety in schools 
according to student reports is to form composite scales that combine reports for multiple items. 
Results for such scales are displayed in a T-score metric - where the mean is 50 for the nation’s 
schools and the standard deviation is 10 - are shown in Appendix Table H2.17. The mean score 
for Safety according to student reports for urban middle/junior high schools is a standard 
deviation below the mean for all schools (T-score = 39.9). The mean score for student 
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Table 2.9 



Percentage o/~ Students. Experiencing Specific Threat s or Violence^ This Year in School 



Experience 


Percentage 


95% Cl 


N 


Seen a teacher threatened by a student 


28 


26.5 - 30.2 


15965 


Had to fight to protect yourself 


20 


18.9 - 21.8 


15974 


Seen a teacher hit or attacked by a student 


12 


10.4 - 12.9 


15966 



Note. Percentage = weighted percentage. 95% Cl = 95% confidence interval for percentage. 
N = unweighted number of respondents. 



Victimization for urban middle schools is more than three quarters of a standard deviation above 
the mean for all schools (T-score = 57.7). Once again, middle schools are seen as less safe and to 
be characterized by more victimization than are high schools. 

Discussion and Summary 

The NSDPS is a valuable source of contemporary information about problem behavior in 
schools. At the same time, any single research project has limitations and ambiguities. In this 
section, some of these limitations are discussed. This section also discusses the nature of 
problem behavior in schools and emphasizes the variability of problem behavior among schools. 

Difficulties in Obtaining Information About Problem Behavior 

The most important single limitation in interpreting information about problem behavior 
provided by the NSDPS stems from the difficulty that was encountered in obtaining the 
cooperation of schools and school districts with the research. Participation rates for principal, 
teacher, and student surveys were described in Chapter 1 (Tables 1.2, 1.4, 1.5, and 1.6). A school 
is considered to have participated in the teacher or student surveys only if a sufficient number of 
questionnaires of each type was returned to represent a usable response. The highest level of 
participation was obtained for the Phase 1 principal questionnaire, where 66% of schools 
participated. Few schools that failed to participate in Phase 1 participated in the Phase 2 
principal survey, and the participation rate for the Phase 2 principal survey fell to 50%. Even 
fewer schools participated in the portion of the research involving surveys of teachers (46%) and 
students (36%). 

Participation was more difficult to obtain among urban schools and it was particularly 
difficult to obtain in urban high schools. Whereas 75% of rural elementary schools participated 
in the Phase 1 principal survey, only 59% of urban high schools participated in that survey (see 
Table 1.2). A usable level of participation in student surveys was obtained in 50% of the rural 
middle/junior high schools from which it was sought, whereas only 23% of urban high schools 
participated in student survey (see Table 1 .5). Participation was associated with a number of 
school and community characteristics summarized in Appendix Table B 1 . 1 . The school 
characteristics examined in Table Bl.l are estimates obtained from the mailing list vendor or 
from the Common Core of Data (National Center for Education Statistics, n.d.). The community 
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characteristics examined there are for the zip code aggregations of 1990 census data (Bureau of 
the Census, 1992, 1993). Several indicators imply that schools in central cities of urbanized 
areas were less likely to participate than were rural or suburban schools. Elementary and 
middle/junior high schools were more likely to participate if located in areas where most housing 
is owner occupied. There is also a tendency for elementary and middle schools located in areas 
where a high proportion of families with children are female headed to be under represented 
among participating schools. High schools in areas where residents are highly educated are 
under represented among participating schools. 

Although this was a study supported initially by the National Institute of Justice and 
eventually by the U.S. Department of Education as well, and although it was endorsed by the 
National Association of Secondary School Principals and the National Association of Elementary 
School Principals, it was very difficult to recruit schools to participate. Differential participation 
rates for schools in communities with different characteristics may have introduced bias into 
some estimates, and it is not possible to know precisely how much bias may have been 
introduced. Although school weights were designed to minimize bias by correcting for some 
differential response rate tendencies, and although the rationale for their use is plausible, their 
potential effect on bias cannot be known. 

Refusal to participate occurred both at the school and at the district level. In most cases, we 
do not know why schools refused to participate, because principals or other educational 
personnel were adept at avoiding our attempts to communicate. And, of course, we can never 
know for sure why specific principals or school districts refused to participate as they were at 
liberty not to tell us or to tell us anything they found convenient. 

Influence of school reluctance to participate on the research. The first influence that 
reluctance on the part of schools or districts had on the research was the large amount of effort 
that had to be devoted to persuading schools or districts to participate. 

The second influence is the potential for nonparticipation to introduce systematic error (bias) 
in the results. Despite efforts to reduce bias by application of nonresponse weights, there is no 
convincing way to eliminate refusal to participate as a potential source of bias. 

Alternative Ways of Estimating the Extent of Problem Behavior 

Different methods of measuring crime and other forms of problem behavior are expected to 
produce different estimates of its level. For example, estimates of the incidence of crime made 
from counts of reports of crime made to the police are very much lower than the estimates made 
from the self-reports of individuals about crimes they have committed or the reports of citizens of 
their personal victimizations. Some part of the difference in levels estimated by the different 
methods is undoubtedly due to defects of the different methods (Hindelang, Hirschi, & Weis, 
1981). For example, any method that depends on reports of an official (a police officer, a school 
principal) will generally underestimate crime or problem behavior for the simple reason that not 
all instances of such behavior will be known to the official, the official may not regard some 
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behavior as sufficiently serious instances to be recorded or reported, the official may fail to 
report, and so on. 

But some part of difference in levels estimated by different methods is also due to the 
measurement of different things. For example, counts of reported crime measure events that at 
least someone regarded as sufficiently serious to merit being reported. On the other hand, counts 
of self-reported misconduct usually will capture some minor or non-criminal behavior that 
matches the description of the behavior inquired about. Scales composed of collections of self- 
reports about a variety of behaviors have the virtue that they can sample from a broad band of 
problem behavior and may be most valuable for measuring individual differences in propensity 
to misbehave. Some of the misbehavior, however, would not always be defined as “crime.” 
However that may be, Hindelang et al. (1981) concluded from a careful study of the reliability 
and validity of self-reports of delinquent behavior that “delinquency exists most clearly in the 
minds of those least likely to engage in it” (p. 219) and that self-reports of delinquent behavior 
may be least valid for those groups who are most delinquent. 

Similar measurement issues occur when victim reports (Sparks, 1982) or different ways of 
measuring drug use or abuse are used as indicators (Reuter, 1999). A person must interpret an 
event as a victimization in order to report it, and it is evident that what is seen as a criminal 
victimization to one person may not be interpreted that way by another. Accordingly, it may be 
best to interpret victim reports as reports of perceived incivilities or crimes. 

All estimates of problem behavior or crime based on reports in questionnaires or interviews 
(as opposed to archival records) are prone to be subject to errors related to the decay of memory 
as well as to the telescoping of events outside of a recall period into the recall period - and some 
respondents may not attend much to a recall period at all. It is not expected that rates estimated 
for a one-month recall period will translate in any straightforward way into rates estimated for 
other recall periods - e.g., one year. 

For all of these reasons it is expected that different indicators will produce different 
estimates of the amount and possibly the distribution of problem behavior. Each of the indicators 
based on principal, teacher, and student reports described earlier in this chapter are of separate 
value. 

Other Surveys of School Crime and Disorder 

No surveys of crime and disorder in schools that are strictly comparable to the present one 
exist. 

Fast Response Survey System. One superficially similar study is a Fast Response Survey 
System (FRSS) study conducted by Westat for the National Center for Education Statistics 
(Heaviside, Rowand, Williams, & Farris, 1998) in the spring and summer of 1997. The FRSS 
study polled principals about crimes reported to police or other law enforcement representatives. 
In contrast to the present survey, the FRSS provided respondents with definitions of terms used 
and asked respondents to report incidents involving multiple crimes only once - essentially 
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listing the incident only for the most important type of crime it involved. Whereas the present 
survey depended almost entirely on mailed responses (a handful of questionnaires were 
completed in telephone interviews), telephone interviews were utilized more extensively in the 
much shorter FRSS. The FRSS combination of interview and mailed questionnaire yielded a 
higher response rate (88%) than the did the present survey. The universe for the FRSS was 
limited to public schools and excluded special education, vocational, alternative and ungraded 
schools. The universe for the present study included public, private, and Catholic schools and 
did not exclude special schools. 

The general pattern of results for school crimes are similar in the two studies. Both show 
much more crime in middle schools, that minor crimes are much more common than serious 
crimes, and that there is a tendency for most crimes to occur at higher rates in urban schools than 
in other schools. The present study sometimes found higher rates than did the FRSS, however. 
For example, we estimate (see Table 2.1) that 21% of middle schools had at least one incident of 
physical attack or fight with a weapon reported to law enforcement, whereas the estimate from 
the FRSS was 12%. Similarly, we estimate that 17% of middle schools had at least one incident 
of robbery reported to authorities, whereas the FRSS estimate was 5%. In both cases the 
differences in estimated rates are statistically significant at the/? < .01 level. Such differences as 
these which are beyond what is expected due to sampling error may be due to (a) differences in 
the universe of schools, (b) differences in the way questions are presented, (c) differences in data 
capturing technique (phone interview versus self-report questionnaire), (d) differences in the 
context in which questions are embedded, (e) differences due to increased sensitivity of 
respondents to crime or increased propensity to report such crimes to the police in view of highly 
publicized violent events in schools that occurred between the two surveys, or (f) differences in 
the level of crime occurring between the two survey occasions. If FRSS respondents attended to 
the instruction to count each incident involving multiple crimes only once, this would tend to 
produce lower estimates than would the lack of such instruction. All of these possibilities are 
worthy of further exploration, but they are beyond the scope of the present report. 

Safe School Study. A second survey of a national sample of schools with superficial 
similarity to the present one is the Safe School Study (SSS) conducted for the National Institute 
of Education (1978). The SSS conducted surveys of public junior and senior high school 
students and teachers, excluding those in comprehensive (e.g., K - 12) schools and perhaps 
excluding those from school districts with 50 or fewer students (it is not clear from the report). 

Again, the general pattern of results for school crimes are similar in the SSS and the NSDPS. 
Both show much more crime in middle schools, that minor crimes are much more common than 
serious crimes, and that there is a tendency for most crimes to occur at higher rates in urban 
schools than in other schools. The specific levels estimated by the two surveys sometimes differ, 
however. The SSS conducted surveys at different months during the year, whereas all of the 
NSDPS surveys were conducted in the spring. In addition, there were differences in the way the 
questions were presented with the SSS using a branching format such that respondents were first 
asked about thefts and then about the size of the thefts, about attacks and then about whether a 
doctor was required. In the NSDPS respondents were first asked about victimizations in the 
school year and then asked about victimization in the past month, whereas in the SSS 
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respondents were asked only about specific months. Appendix Table H2.18 shows details for 
two similar questions asked of students in both surveys. The estimated percentage of students 
reporting theft of items worth less than a dollar in the 1976 survey was very much larger than the 
number estimated in the 1998 survey. The percentage of students reporting that they had been 
attacked but not hurt badly enough to see a doctor was also lower in the 1998 survey although the 
differences between 1976 and 1998 rates are not significant for the high school and rural school 
comparisons. Estimated rates of minor theft per 1000 teachers is also lower in the 1998 NSDPS 
than in the 1976 SSS. But the estimated rates of attack not serious enough to require seeing a 
doctor are higher for the NSDPS than for the SSS; the rate for rural schools is very much higher 
in the 1998 than in the 1976 survey. Appendix Table H2.19 shows the details. 

Such differences as these which are beyond what is expected due to sampling error may be 
due to (a) differences in the universe of schools, (b) differences in the way questions are 
presented, (c) differences in the context in which questions are embedded, (d) differences in 
participation rates, or (e) differences in the level of crime occurring between the two survey 
occasions. All four of these possibilities may account for differences in the level of these crimes 
estimated from the NSDPS and the SSS. In the NSDPS the questionnaire contained separate 
questions about minor attack (no doctor) and more serious attack (doctor attention required) and 
separate questions about minor and more serious theft, whereas in the SSS these questions were 
not independent. Potentially of equal importance, the “last month” victimization questions in the 
NSDPS questionnaire were preceded by a series of questions asking about victimizations 
occurring in the last year. It is possible that the longer reference period in the preceding 
questions in NSDPS led some respondents to infer that researchers were inquiring about more 
serious incidents (Winkielman, Knauper, & Schwarz, 1998). This “more serious” set may then 
have carried over to the “last month” questions. The way questions are asked and the context 
within which they are embedded can influence estimates produced from them (Krosnick, 1999; 
Schwarz, 1999). 

School Crime Supplement to the National Crime Victimization Survey. A third study with 
some similarity to the present one is the School Crime Supplement (SCS) to the National Crime 
Victimization Survey (NCVS), conducted in 1989 and again in 1995. In the SCS (Chandler, 
Chapman, Rand, & Taylor, 1998) a household survey with a 6-month recall period is conducted, 
and household members between the ages of 12 and 19 who attended a school at any time in the 
past six months and who were enrolled in a school that could lead to a high school diploma were 
asked supplemental school-related questions after completing the NCVS interview. Unlike the 
SSS and NSDPS, students are located in the SCS by going through sampled households rather 
than sampled schools, and students in any kind of school (not just public) would be included if it 
could be on the pathway to a high school diploma. Over 70% of the SCS respondents were 
interviewed by telephone with most of the rest interviewed in person, and some were interviewed 
by proxy (i.e., someone else provided information about the member of the sample). The SCS 
asked students whether they avoided certain places in the school that are similar to questions 
asked in the NSDPS, but there are minor wording changes and the SCS implied a 6-month recall 
period and asks if the places were avoided. The NSDPS asks instead if a student usually avoids 
these places. 
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A similar pattern of results were obtained in the SCS (Kaufman, Chen, Choy, Chandler, 
Chapman, Rand, & Ringel, 1998) and the NSDPS. In both studies, younger individuals are more 
likely to have avoided places at school. Black and Hispanic students were more likely to report 
avoiding places than were White students, and students in urban areas were more likely to avoid 
places in school than other students. At the same time. Appendix Table H2.20 shows that the 
percentages of students who reported that they usually avoid places in school in the NSDPS are 
much larger than the percentages who reported that they did avoid places in the school in the 
SCS. For all students aged 12 or older, 20% reported usually avoiding at least one of five 
specific places in school in the NSDPS whereas 9% reported avoiding these same places in the 
1995 SCS. Again, the context within which questions are asked may help explain the large 
differences in the percentages obtained in the two studies. In the SCS, individuals were asked 
about school experiences after having completed a lengthy survey of general crime victimization 
questions. There is no way at this time to determine what features of question presentation, 
interview versus self-report approach, question context, or differences over time may account for 
the large differences in level of avoidance of places in school estimated in the two studies. 

It is probably best to regard any single estimate of the level of problem behavior, 
victimization, or safety as a function in part of school safety and in part of method of inquiry. 6 
Put another way, alternative indicators should be viewed as alternative indicators and no 
indicator viewed as providing an absolute count of problem behavior, victimization or safety. 
Because alternative indicators generally show the same pattern of results across groups of 
respondents or schools, it appears sensible to make comparisons among schools or among 
individuals within any of the studies mentioned here. But it does not appear profitable to 
speculate too much about the meaning of differences in levels estimated according to different 
methods. 

Other Limitations of the Information from the NSDPS 

Sampling. All survey search shares the limitation that the information developed depends 
upon the validity of the reports of respondents. It is well known that respondents make errors of 
interpretation and recall in reporting events such as personal victimization (Panel for the 



6 Alternative sources of information about youth problem behavior exist. These include (a) the 
Monitoring the Future surveys (Johnston, O’Malley, & Bachman, annually) which have made 
annual inquiries of a national sample of high school seniors since 1975 (and since 1991 of eighth 
and tenth graders as well); (b) the Youth Risk Behavior Surveillance summaries (Kann et al., 
1998) conducted in odd-numbered years since 1991 in school-based surveys in 33 states and 
certain localities including student self-reports of fighting, carrying weapons, feelings of safety at 
school, and other problem or risky behavior; (c) the National Longitudinal Surveys of Youth 
1997 cohort (Bureau of Labor Statistics, 1999) in which persons aged 12-16 years identified in a 
household screener survey were asked to complete a self-administered questionnaire about drug 
use and delinquent behavior. We have compared the present survey with those that appear on 
their face to provide the most comparable data. See Appendix Tables H2.21 and H2.22 for more 
on self-reported delinquent behavior from NSDPS. 
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Evaluation of Crime Surveys, 1976), the way questions are asked influence the answers 
(Krosnick, 1999), and that people do not always tell the truth. Error could also enter the data 
when respondents make mistakes in marking answer documents, or when an interviewer makes 
an error in recording information. Accordingly, the information gleaned from the present surveys 
should be regarded as one fallible source of information about the matters explored in this report. 

Most survey research is also limited by the size and nature of the sample examined. In the 
present research, our aim was to obtain a probability sample of all schools serving students in any 
grade from kindergarten through grade 12, including public, Catholic, and private schools. 
Coverage error is present to the extent to which the list from which our sample was drawn is 
incomplete. Although we used what we judged to be the most up-to-date and complete list of 
schools available, some degree of coverage error is probably inevitable. Error or bias may occur 
when schools or individuals fail to participate in the provision of information. This source of 
error or bias was described above. Although non-response adjustments were used to minimize 
the effects of non-participation error or bias, these adjustments are probably imperfect. 

Because it is prohibitively expensive and unduly burdensome of respondents to include ail 
schools and all individuals in the present research, samples were selected to represent all of the 
schools in the nation. Within schools, samples of students were drawn to represent the school. 
Weights have been applied to responses so that estimates from the probability sample represents 
all schools, and as noted these weights have been adjusted to account as well as is possible for 
non-participation. Estimates made from samples naturally differ somewhat from the values that 
might be obtained from a complete enumeration. In this report, standard errors or confidence 
intervals are usually presented to provide indices of the variation due to sampling that may be 
expected. 7 Readers are encouraged to consider the point estimates and confidence intervals (or 
standard errors) in interpreting information. 

Raters are nested within schools. In this research, all respondents were asked to report 
about one and only one school. No informant described multiple schools, and it is possible that 
many of the respondents have a limited experience of the full range of schools. When a 
respondent is asked to indicate if gangs are a problem in the school (principal questionnaire), 
how safe from vandalism, attacks, and theft the hallways are (teacher questionnaire), or whether 
the school rules are fair (student questionnaire), the information provided about the school is 
confounded with the perspectives of the respondents. Because a set of schools is not being rated 
by a common set of raters, it is possible that objectively safer schools are sometimes judged less 
safe than objectively less safe schools (Bimbaum, 1999). Students with no experience of very 
safe schools may, for example, indicate that their own disordered school is very safe. Teachers 



’Because a complex sample design involving stratification was used, and because of the use of 
weights adjusting for the sample design and nonparticipation, standard errors have been 
estimated using a re-sampling method known as the jackknife. Because the estimation of 
standard errors in this way is time consuming and cumbersome, some appendix tables report 
“nominal” significance levels, which are based on estimates made on the assumption of simple 
random sampling which underestimates sampling error. 
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and principals may do the same. Worse, principals in some schools experiencing a great deal of 
disorder may tend to discount the seriousness of many specific incidents of crime or violence and 
report relatively fewer incidents than may principals in very orderly schools. In the case of 
teacher and student reports, it is possible to estimate the proportion of variance that lies between 
schools (see Appendix F), because there are multiple raters for each school. Even so, the fact 
that raters are nested within schools and probably do not have direct experience of the full range 
of school environments is an inherent limitation in survey research of the present kind. 

Summary 

The nature of problem behavior in schools. Minor forms of problem behavior are common 
in schools. For example, 27% of teachers report that student behavior keeps them from teaching 
a fair amount or a great deal. This minor misconduct can be a serious problem because it 
interferes with efforts by schools to pursue their mission to conduct education. The percentage of 
teachers per school reporting that student behavior keeps them from teaching at least a fair 
amount ranges from 0% to 1 00%. In a quarter of schools 42% or more of teachers-report that 
student behavior keeps them from teaching at least a fair amount. 

Serious forms of problem behavior such as physical attacks or fights involving a weapon, 
robberies, or treats involving a knife or a gun occur less frequently than the more pervasive 
minor kinds of student misconduct. But they occur frequently enough that they are also clearly 
major problems. Almost 7% of schools reported at least one incident of physical attack or fight 
involving a weapon to law enforcement officials, and for middle/junior high schools the 
percentage was 21%. Being threatened or attacked in school is a relatively common experience 
among students, with 19% of students reporting threats and 14% reporting attacks. A startling 
5% of students report having been threatened with a knife or a gun. Such incidents are far less 
common among teachers. Although 20% of secondary school teachers (and 31% of urban 
middle school teachers) report being threatened in remarks by a student, half of one percent 
report having had a weapon pulled on them and seven tenths of a percent report having been 
attacked and having to see a doctor. 

Evidence from the reports of teachers, principals, and students implies that most kinds of 
problem behavior are more common in middle schools than in elementary schools or high 
schools. The exception is drug use - student self-reports imply that drug use is more extensive in 
high schools. 

Variability among schools. There is variability among schools in the level of crime or 
disorder they experience. According to the school crime scale - which indexes the extensiveness 
of a variety of crimes reported to law enforcement according to principal reports - the average 
urban middle/junior high school scores about a standard deviation for schools above the mean for 
all schools (Table 2.2). Equally important, there is great variability among urban middle/junior 
high schools in their scores on the school crime scale. 

One way to obtain a concrete impression of the degree of variability observed among schools 
is to review the reports of principals, teachers, and students for four school shown in Table 2.10. 
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This table shows how the people in two urban and two suburban middle schools described the 
school environment. The schools were selected so that there is one relatively safe and one 
relatively less safe school in each type of location according to the Student and Teacher Safety 
scales. The Teacher Safety scale T-scores for the four schools are as follows: A = 62, B = 35, C 
= 55, D = 34. The Student Safety scale T-scores for the four schools are A = 67, B = 3 1 , C = 53, 
D = 29. The Teacher Victimization T-scores are A = 38, B = 68, C = 52, D = 72. The Student 
Victimization T-scores are A = 37, B = 65, C = 53, D = 61 . These are not the most extreme 
schools in the sample, but they illustrate the variation. 

Table 2.10 shows that school B’s principal indicated having reported 40 physical attacks or 
fights without a weapon to law enforcement personnel, school C indicated having reported 1 0, 
school A reported 0, and school D failed to provide this information. The majority of teachers in 
the two less safe schools report that students often or almost always talk at inappropriate times, 
make disruptive noises in class, tease other students, make threats or curse at others, and are 
distracted by student misbehavior. Much smaller percentages of teachers in the relatively safer 
schools report that these kinds of misconduct occur often or almost always. In the relatively safer 
suburban middle school, 9% of teachers report that the behavior of some students keep them 
from teaching a fair amount or a good deal of the time; in the relatively less safe urban middle 
school, 74% of teachers report being blocked from teaching by student behavior. In the two less 
safe schools 72% and 74% of teachers indicated that they received obscene remarks or gestures 
from students; in the safer schools the percentages were 6% and 31%. Over half of the teachers 
reported having been threatened by a student in the two less safe schools, whereas only 0% and 
6% of teachers in the two safer schools reported such treats. Students’ reports of victimization 
experiences in the safer and less safe schools are not as great as might be expected. Schools that 
score high in safety by one criterion do not always score high according to other criteria. 

The concrete portraits provided by examining the details of these four schools’ reports of 
crime, victimization experiences, classroom orderliness, and perceptions of safety underscore the 
earlier characterization of disorderly schools as uncivil places. Incivility appears to be more 
pervasive than the most serious kinds of crimes such as attacks involving weapons. Physical 
attacks and fights, however, are not rare in schools. 
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Table 2.10 



Illustrative Middle Schools Differing in Their Levels of Safety 





Suburban 


Urban 






Less 




Less 




Safer 


safe 


Safer 


safe 


Source and school characteristic 


A 


B 


C 


D 


Principal 










School enrollment 


535 


1230 


264 


1013 


Number of crimes reported to authorities 










Physical attack or fight, weapon 


0 


0 


0 


0 


Physical attack or fight, no weapon 


0 


40 


10 


NR 


Robbery 


0 


0 


0 


0 


Theft or larceny 


0 


10 


12 


NR 


Vandalism 


0 


5 


15 


NR 


Teachers (% saying often or almost always) 










Students pay attention in class 


97 


59 


75 


39 


Students take things that do not belong to them 


0 


31 


0 


61 


Students do what I ask them to do 


97 


69 


94 


48 


Students destroy or damage property 


3 


25 


0 


55 


Students talk at inappropriate times 


21 


70 


25 


81 


Students make disruptive noises 


0 


54 


19 


58 


Students try to physically hurt other people 


0 


41 


12 


39 


Students tease other students 


6 


80 


38 


65 


Students make threats to or curse at others 


3 


54 


6 


61 


Students are distracted by the misbehavior of other 
students 


6 


75 


25 


78 


The classroom activity comes to a stop because of 
discipline problems 


3 


34 


13 


52 


I spend more time disciplining than I do teaching 


0 


31 


13 


39 



continued . . . 
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Table 2.10 (continued) 

Illustrative Middle Schools Differing in Their Levels of Safety 





Suburban 


Urban 






Less 




Less 




Safer 


safe 


Safer 


safe 


Source and school characteristic 


A 


B 


C 


D 


Teacher (% responding a fair amount or a great deal) 










How much does the behavior of some students in your 
classroom keep you from teaching? 


9 


65 


38 


74 


Teacher (% experiencing in school year) 










Damage to personal property worth less than $10 


3 


51 


56 


36 


Damage to personal property worth more than $10 


3 


23 


31 


23 


Theft of property worth less than $10 


6 


41 


31 


32 


Theft of property worth more than $ 1 0 


6 


12 


12 


36 


Was attacked and had to see a doctor 


0 


3 


0 


10 


Was attacked, not seriously enough to see a doctor 


0 


15 


0 


16 


Received obscene remarks or gestures from a student 


6 


72 


31 


74 


Been threatened in remarks by a student 


0 


59 


6 


58 


Had a weapon pulled on me 


0 


0 


0 


0 


Teacher (% indicating very unsafe or fairly unsafe) 










Your classroom while teaching 


0 


10 


0 


23 


Empty classrooms 


0 


21 


0 


28 


Hallways and stairs 


0 


34 


0 


19 


The cafeteria 


0 


25 


0 


26 


The restrooms used by students 


0 


32 


0 


29 


Locker room or gym 


7 


51 


0 


14 


Parking lot 


3 


37 


0 


19 


Elsewhere outside on school grounds 


0 


12 


0 


19 



continued . . . 
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Table 2.10 (continued) 

Illustrative Middle Schools Differing in Their Levels of Safety 



Suburban Urban 



Less Less 



Source and school characteristic 


Safer 

A 


safe 

B 


Safer 

C 


safe 

D 


Students (% experiencing in school year) 


Theft of less than $1 from locker or desk 


42 


64 


60 


71 


Theft of greater than $ 1 from locker or desk 


49 


60 


36 


67 


Physical attack 


2 


22 


30 


14 


Robbery, things worth less than $1 


0 


8 


6 


6 


Robbery, things worth more than $1 


0 


8 


8 


8 


Threat of beating 


0 


26 


17 


15 


Threat with knife or gun 


0 


2 


4 


6 


Students (% avoiding place) 


Shortest way to school or the bus 


0 


28 


11 


32 


Any entrances into the school 


0 


22 


6 


24 


Any hallways or stairs in the school 


0 


8 


6 


24 


Parts of the school cafeteria 


5 


26 


6 


22 


Any school restrooms 


2 


17 


9 


20 


Other places inside school building 


0 


19 


0 


26 


Other places on the school grounds 


0 


20 


4 


20 


Students (% experiencing or observing this year) 
Had to fight to protect yourself in school 


0 


33 


22 


28 


Seen a teacher threatened by a student 


0 


50 


28 


44 


Seen a teacher hit by a student 


5 


36 


6 


38 



Note. NR = no report; number not ascertained because principal made no report. 
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Activities to Create and Maintain Safe and Orderly Schools 



The public expects today’s schools to do many things. Among these are the implementation 
of a wide variety of approaches to reducing problem behavior, improving discipline, and 
promoting safer schools. In this chapter, we undertake the task of describing what schools do to 
prevent problem behavior. We examine the range of what is undertaken and how much activity 
is undertaken. First, we put the contemporary requirement that schools improve youth behavior 
in historical context by describing the extension of schooling to a large percentage of youths who 
would not have received much schooling in the past. 

The Press for Delinquency Prevention Activity in Schools 

At one time, the family was the main source of occupational learning (Coleman, 1972). As 
recently as 1930, 70% of children lived in two-parent farm families (Hernandez, 1994). In 1940, 
10% of children lived with a mother in the paid labor force, but by 1990 60% of children had a 
mother in the labor force. In 1900, the number of high school graduates as a percentage of the 
18-year-old population stood at about 6%; by 1970 this had reached 78% (Cartter, 1976). G. 
Gottfredson (1981) documented changes over time that imply a decreased involvement of young 
people with work roles and with adults outside of school, and an increased involvement of ever 
larger proportions of youth in school for ever larger numbers of days per year. In 1 870 the 
average length of school terms was 132 days and the average number of days of school attended 
was only 78 days (President’s Science Advisory Committee on Youth, 1973). Today school 
terms are usually 180 days. Even in large urban school districts that are notorious for attendance 
problems, average daily attendance rates of about 80% are reported (Council of Great City 
Schools, 1 994), and in some school districts average daily attendance is 95% or more (South 
Carolina Department of Education, 1997). This means that students today attend school an 
average of 144 to 171 days. Between 1940 and 1996 the percentage of the population aged 25 to 
29 years that had completed 4 years of high school or more increased from 38% to 87% (National 
Center for Education Statistics, 1998b). 1 In short, a strikingly larger range of today’s youths are 
involved in school and attend school much more of the time than was true in the past. 

Along with this shift away from family as a source of occupational learning and the 
participation of a greater range of young people in schools have come, not surprisingly, calls for 
schools to do more things. Schools are called upon to go beyond the development of traditional 



'By 1996, four years of high school had been completed by 93% of the White non-Hispanic 
population, 86% of the Black non-Hispanic population, and 61% of the Hispanic population aged 
25 to 29 years. The Black high school completion rate in 1996 was about equal to the White 
completion rate in 1975; the Hispanic completion rate in 1996 was about equal to the White 
completion rate in 1956. Although the race/ethnic group disparity in high school completion 
remains alarming, the proportions of persons in schools that are Black or Hispanic have shown 
large historical increases (see National Center for Education Statistics, 1998b). 
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academic or intellectual skills to the development of vocational skills, decision-making skills, 
skills for coping with employers and organizations, skills required to avoid undesirable social 
pressures from others, and competencies in making long-range plans and delaying gratification. 
Schools are called on to play a role in the socialization of the young for participation in an 
orderly civil society and in their own orderly education. Now, the vast majority of youths are 
expected to complete high school. Dropping out (leaving school before completing high school) 
even for youths who do not do well in school, do not like school, or who do not behave well in 
school is usually seen as an undesirable outcome. 

The public appears to want schools to do a better job of discipline. In occasional opinion 
polls conducted between 1970 and 1998, the percentage of respondents indicating that lack of 
discipline is a major problem facing the local public schools has ranged from 14% to 27% 
(National Center for Education Statistics, 1998b; Rose & Gallup, 1998). In the most recent Phi 
Delta Kappa/Gallup poll, 20% of public school parents cited fighting, violence, or gangs as the 
most important problems facing the schools. 

All of these developments have led schools to attempt a wide variety of approaches to 
reducing problem behavior, improving discipline, and promoting safer schools. Developing a 
description of the delinquency and other problem behavior prevention activities of schools 
required first that we develop definitions of the activities we sought to describe and second, that 
we develop a classification that would provide a system and a vocabulary for discussing these 
activities. Ultimately, these definitions and the taxonomy will be useful if they contribute to an 
understanding of which kinds of.activity are helpful and which are not. These definitions and the 
development of a taxonomy of prevention activity are described next. 

Definitions 

A prevention program is defined as an intervention or set of interventions put in place with 
the intention of reducing problem behavior in a population. Such activities include — but are not 
limited to — policies, instructional activity, supervision, coaching, and other interventions with 
youths or their families, schools, or peer environments. Problem behaviors include criminal 
behavior; alcohol, tobacco, and other drug use; and risky sexual activity. Prevention programs 
may target these problem behaviors directly, or they may target individual or social 
characteristics believed by program advocates to be precursors of problem behavior. These 
individual and social characteristics include, but are not limited to, poor social competency and 
related skills, impulsiveness, academic failure, limited parental supervision, harsh or erratic 
discipline, poor classroom management, or ineffective school or community guardianship. 
Because we are concerned with what schools are doing, we limited the search to school-based 
prevention activity. By this we mean activity that is primarily located in a school building or that 
is implemented by school staff or under school or school-system auspices. Kindergarten, 
elementary, and secondary school levels are included. Elaboration of our definitions and 
rationales for them are provided in Appendix D. 



Development of a Taxonomy of Practices, Programs, and Arrangements 

To conduct research on what schools do to prevent delinquent and other problem behavior 
and to promote a safe and orderly environment, we required a useful classification of school 
activities or programs and a classification of program objectives. Classifications are useful 
because they organize related activities together, make communication about activities easier, aid 
in recall, and distinguish unlike activities or objectives by classifying them separately (Sokol, 
1974). 2 

A first step in developing the classification was to conduct a search to identify the full range 
of activities that would have to be classified (Womer, 1997, provided an earlier account of this 
effort). We scoured the scientific and practitioner literatures to learn about the universe of 
prevention programs and practices. A search of existing school-based prevention strategies was 
conducted to discover the full range of prevention activities in schools and to ensure that the 
taxonomy to be used in this research was as comprehensive as possible. This search revealed a 
wide variety of programs including well-known and widely disseminated programs and practices 
such as Drug Abuse Resistance Education (D. A.R.E.), Law Related Education (L.R.E.), and 
Midnight Basketball programs. This search also discovered programs that used unusual 
prevention methods such as lacrosse, clown troupes, or planting trees to combat violence and 
drug use. 

This section describes the method of program retrieval for this activity and the taxonomy of 
programs, practices, and arrangements that emerged from this activity. 

Sources Used to Obtain Leads 

An initial search located 513 school-based prevention programs sponsored by government 
agencies, foundations, and school systems. Table 3.1 displays the variety of sources that were 
used to obtain leads to specimens of program or activities. Among these sources were lists of 
federal and state grant recipients including those from the National Institute of Drug Abuse 
(NIDA), the National Institute of Justice (NIJ), the Center for Substance Abuse Prevention 
(CSAP), and Community Schools grantees. Foundation grants lists were also obtained from the 
University of Maryland's Office of Research Advancement and Administration, Youth Today, the 
W. K. Kellogg Foundation, and the William T. Grant Foundation. Additional sources include 
published literature and technical assistance resources from various agencies and publishers and 



Prevention activities can, in principle, be categorized in many ways. Some of these are the age 
or grade of the target population, the specific problem behavior in which they focus, their 
intermediate objectives, or the nature of the activity undertaken. 
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source materials cited in secondary accounts. Referrals from persons contacted were a final 
source for leads. 



Table 3.1 

Sources^ Used_ to Obtain Leads 

Federal and State Grant Recipients 

• Center for Substance Abuse Prevention 

• Community Schools 

• National Institute of Drug Abuse 

• National Institute of Justice 

Foundation Grants Lists 

• University of Maryland's Office of Research Advancement and Administration 

• Youth Today 

• W.K. Kellogg Foundation 

• William T. Grant Foundation 

Technical Assistance Resources Searched 

• Administration on Children, Youth, and Families 

• Appalachian Educational Laboratory 

• Carnegie Council of New York 

• Center for Disease Control 

• Coordinating Council on Juvenile Justice and Delinquency Prevention 

• Center for Research in Educational Policy 

• Center for Substance Abuse Prevention 

• Drug Strategies 

• Educational Development Center 

• National Criminal Justice Research Service 

• National School Safety Center 

• North Carolina Center for the Prevention of School Violence 

• Northwest Regional Education Laboratory 

• Office of Juvenile Justice and Delinquency Prevention 

• South Eastern Regional Center for Drug-Free Schools and Communities 

• South Eastern Regional Vision for Education 

» U.S. Department of Agriculture 



Methods Used to Obtain Information About Programs or Activities 

Telephone calls were the primary method of obtaining information from program sources. 
Phone calls were made for approximately four months to all organizations and agencies 
identified as operating prevention programs. The calls requested written program descriptions, 
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evaluation reports, implementation manuals, or other materials describing their school-based 
program. After four months of phone calls, letters were mailed to each agency which had not 
been reached by phone or from which written information was not yet received. These letters 
explained the purpose of the study and requested a written description of their program. 

Program materials were reviewed to determine whether they met our selection criteria. 
Inclusion criteria required that a program take place within the school building or under school 
auspices and that the program be intended to prevent problem behavior or promote school safety. 
Each program meeting the criteria was coded for discrete program activities according to the 
classification of school-based prevention activities developed for this project. 

What Happened When Materials Were Sought 

Written materials were obtained for 35% of the 513 leads. Of the 178 program descriptions 
that were obtained, 78% (N = 139) met the selection criteria. The remaining 39 programs did not 
fit our definition of "school-based" or were too vague to classify and were, therefore, not 
included in the final sample. 

Most prospects identified by searching the lists and technical assistance reports led to dead 
ends. There are several reasons why the majority of leads resulted in a dead-end. In many cases 
the person responsible for disseminating information about the program could not be reached. In 
some instances the program was no longer in existence or the contact person did not have written 
materials. Over one dozen request letters were returned due to incorrect or unknown addresses. 
Finally, program materials were simply not received in one-third of the cases where program 
contacts agreed to send them. 

The materials uncovered by using the foregoing method, together with program materials we 
had acquired or knew about as a result of working in the delinquency prevention area for many 
years, were used to construct a classification of program types. Many programs have multiple 
components that resemble more than one category in a classification. 

The Taxonomy 

This work helped to develop a comprehensive classification of prevention activities in 
schools consisting of 24 categories and nearly 300 subcategories. We sought to provide a 
category to describe each important aspect of any problem-behavior-prevention program (in other 
words, to provide an exhaustive set of categories). Our aim was to provide a set of descriptors 
for prevention activities each of which falls in one and only one category (exclusiveness). The 
taxonomic principles or rules for identifying an activity as an instance of a type were spelled out 
in a brief statement, so that identifying a program or activity by category name should provide an 
efficient method for communicating about the program’s characteristics. The development of the 
taxonomy involved an iterative process as we tried to identify instances of specific prevention 
activities using the emerging classification. 
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We wished to distinguish the objectives of an activity or program from the characteristics of 
the activity or program itself. Therefore, a separate classification of potential objectives was 
developed. The complete taxonomies for activities and for objectives are shown in Appendix D. 
Table 3.2 summarizes the classification of activities by listing the major categories. Both of 
these classifications can be supplemented by other classifications - e.g., age or ethnic group of 
target population. 



Table 3.2 

A Classification of Prevention Activity 

0 Information 

1 Prevention curriculum, instruction, or training 

2 Use of cognitive-behavioral or behavioral modeling methods of training or instruction 

3 Behavioral or behavior modification interventions not specified above 

4 Counseling/social work/psychological/therapeutic interventions not specified above 

5 Individual attention interventions not specified above 

6 Recreational, enrichment and leisure activities not specified above 

7 Referral to other agencies or for other services not specified above 

8 Interventions that change instructional or classroom management methods or practices not 
specified above 

9 Interventions that change or maintain a distinctive culture or climate for inhabitants’ 
interpersonal exchanges - or communicate norms for behavior 

1 0 Intergroup relations and interaction between the school and community or groups within the. 
school 

1 1 Rules, policies, regulations, or laws about behavior or discipline or enforcement of such 

1 2 Interventions that involve a school planning structure or process - or the management of 
change 

1 3 Reorganization of grades, classes, or school schedules 

14 Security and surveillance interventions within school and boundary - except school 
uniforms 

1 5 Interventions that exclude weapons or contraband, except rules disallowing weapons or 
contraband 

1 6 Interventions to alter school composition 

1 7 Family interventions (other than home-based reinforcement) 

1 8 Training or staff development intervention not specifically directed at an intervention 
specified above 

1 9 Removing obstacles or providing incentives for attendance 

20 Architectural features of the school 

21 Treatment or prevention interventions for administration, faculty, or staff - or employee 
assistance programs 

22 Other intervention not specified above 

23 Not specified intervention 



The taxonomies of activities and objectives were constructed to provide for the classification 
of programs that were observed, not just theoretical programs. For this reason they allow for the 
classification of activities that may have little or no plausibility as approaches to reducing 
problem behavior. For example, there is scant reason for believing that the provision of a modest 
amount of recreational activity will take a big bite out of crime. But there are many such 
programs being operated that are regarded by those who operate or support them as delinquency 
or drug prevention programs. Similarly, there is little evidence that would suggest that targeting 
low self-esteem or alienation will be fruitful approaches to the prevention of problem behavior 
(D. Gottfredson, Harmon, Gottfredson, Jones, & Celestin, 1999). But many who operate 
programs believe that (a) their programs will increase self-esteem and (b) that this is a useful 
route to the prevention of problem behavior. To study such activities, they must be classified, 
and so they are included in our taxonomy. 

The relation of some categories in the taxonomy to problem behavior is obvious. For 
instance, instruction in ways to avoid problem behavior, behavior modification, or the use of 
rules and disciplinary practices are linked to the prevention or reduction of problem behavior in 
an obvious way. But we know from making presentations about this research in progress that 
some persons are puzzled by some of the categories in the classification, so it is useful to 
consider briefly how some activities that fall within the categories are related to the prevention of 
problem behavior. Criminologists sometimes ask why improvements to classroom management 
or instruction might be related to delinquency. One answer is that disorderly classrooms provide 
opportunities for students to get into trouble, that school safety and classroom orderliness are 
correlated (G. Gottfredson, 1984/1999). Disorderly classrooms may also contribute to the 
development of patterns of delinquent behavior by making disruptive behavior salient and 
providing visible social rewards for such behavior. Interventions to improve instruction or 
classroom management have been found in some research to produce reductions in problem 
behavior (D. Gottfredson, 1997; D. Gottfredson et al., in press). 

Criminologists also sometimes ask why school reorganization could be related to 
delinquency. Educators sometimes arrange schools into smaller units, for example forming 
schools within a school or separating the grade levels in different parts of the school or on 
different floors, to help reduce problem behavior. It is common, also, for some middle schools to 
have separate stairways for students in the different grades, and many believe that this reduces 
problem behavior. One rationale sometimes offered for such practices is that the smaller groups 
produced bring each adult into continuing contact with a smaller number of students, whom they 
can more easily recognize and who may become more attached to the adults. Class schedules are 
sometimes arranged to give students less time between classes, more time between classes, or 
have different groups of students in the hallways, playgrounds, or eating areas at different times, 
thus reducing opportunities or provocation for fighting or other problem behavior. Such 
arrangements may reduce problem behavior (D. Gottfredson et al., in press). 

Architectural design features of schools may be related to school safety and the prevention of 
problem behavior in part because of the opportunities they provide for surveillance of activity in 
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the school. Some schools are designed so that all persons entering the school are easily visible 
from the school office, and all of the hallways can be observed from a point near the office door. 
Others are built with multiple entrances not visible from the office; some schools are built with 
four stories and a rectangular arrangement of hallways so that observing all hallways would 
require 8 observers. Efficient architectural arrangements for promoting security were described 
by Bentham in 1791 ( Panopticon or the Inspection House) and 1798 {Proposal for a New and 
Less Expensive Mode of Employing and Reforming Convicts) (see Bentham, 1995, and Sample, 
1993). In central cities where school enrollment has declined, some schools wall off portions of 
the building to prevent unobserved access to unneeded space by students or others. 

Finally, reflection will imply that arrangements that alter the composition of the studentry 
are obvious ways to influence school safety and levels of problem behavior. Some schools are 
selective, admitting only students who meet certain academic or behavioral criteria. Others (such 
as some alternative schools or schools for delinquent youths) are intended to serve students who 
display a great deal of problem behavior. Some schools accept the enrollment of students who 
are not wanted in other schools in order to keep their enrollments (and thus staffing levels) up. 

The taxonomy was developed in part by collecting descriptions of programs and practices in 
the manner described earlier in this chapter. The first application of the taxonomy is in 
describing what programs were gleaned through this process. This provides a way of 
summarizing the characteristics of programs that are “marketed” by technical assistance 
organizations, government agencies, and others. This description is presented next. 

Most Common Program Types Marketed 

The 139 marketed program descriptions obtained as a result of our requests (described 
earlier) were classified using the full taxonomy. Programs were coded according to their major 
activities, and each program could be assigned multiple codes if it incorporated activities fal lin g 
in several categories. Sixty-seven percent (67%) of the programs in this sample use group 
instruction, making it the most commonly promoted program feature. Group instruction involves 
teaching students factual information, and sometimes attempts to increase students' awareness of 
negative social influences and prepare them to respond appropriately to harmful situations. 

These programs are often conducted in a classroom setting with teacher lectures, group 
discussions, and demonstrations. Workbooks, worksheets, textbooks, audiovisual materials, etc. 
are often used. Although marketed programs use a variety of program strategies, the 
predominant feature is the prevention curriculum. As will become apparent in a following 
section, this type of activity is also the most commonly used prevention activity in the nation’s 
schools. The marketed programs retrieved in our harvest are described more fully in Table 3.3. 
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Table 3.3 



Percentage of Marketed Programs Using Various Program Features^ 



Percentage 

Program Features Using Feature 


Program Features 


Percentage 
Using Feature 


Group Instruction 


67 


Cognitive Behavioral Training 


18 


Communication of Norms 


29 


Behavior Modification 


11 


Counseling 


26 


Referral to Other Agencies 


9 


Recreational Activities 


25 


Staff Training & Development 


9 


Rules & Regulations 


24 


Changes to School Management 


7 


Individual Attention 


22 


Security & Surveillance 


7 


Family Management Strategies 


21 


Providing Attendance Incentives 


4 


Changes to Classroom 


19 


Exclusion of Weapons & Contraband 1 


Management 




Alteration of School Composition 


1 


Interaction Between School & 
Community 


19 


Reorganization of Grades/Classes 


1 



Note. N= 139. Percentages do not sum to 100 because activities were sometimes classified 
into multiple categories. 



Prevention Activities in the National Sample of Schools 

This section describes the distribution and extent of prevention activities and arrangements 
to reduce or prevent problem behavior or promote a safe and orderly environment in the national 
sample of schools. We obtained initial information about these activities from principals in our 
Phase 1 surveys. Principals completed a screening questionnaire to elicit information about 
activities and arrangements of all types. For fourteen categories of activity, principals were asked 
to name the activity or program and to designate one or two individuals who could provide 
further information. In Phase 2 surveys we obtained additional detailed information from the 
designated individuals about a sample of those programs. The 14 types of activities about which 
we sought detailed information from designated individuals in Phase 2 are more “program-like” 
than the activities, practices and arrangements from which we sought information only from the 
principal. Activities in these 14 categories tend to be more discretionary in nature than the 
school-wide arrangements about which we asked only the principal to report. For example, a 
school may have or not have counseling or a planning team, but all schools have rules, a physical 
environment, and ways of handling discipline. 3 We regarded the principal as the best source of 



3 Some things about which we asked only the principal are also discretionary. Of course it is 
possible for schools to elect not to provide any information about drugs or safety, for example. 
But we elected not to obtain detailed information from designated individuals about the provision 
of information. 



O 

ERIC 



3-9 



information about most school-wide activities and arrangements such as school rules, discipline 
policies, and architectural arrangements. We asked the principal about these arrangements in the 
Phase 1 and 2 questionnaires. 

This section begins by describing school-wide activities and arrangements that are presumed 
to be pertinent in all schools and about which our information is derived from principal 
questionnaires. Then information about the 14 kinds of “discretionary” activities which may or 
may not be applied in schools is summarized. This section focuses on how much activity occurs. 
The following section begins to address the issues of program quality and intensity. 

School-wide Activities and Arrangements 

We asked about certain school-wide activities and arrangements in the Phase 1 Principal 
Screening Questionnaire (completed in Spring, 1997), which asked the principal what activities 
and arrangements to reduce problem behavior or create a safe and orderly environment were 
applied in the school. For example, principals reported on the use of practices that influence 
studentry composition, scheduling practices, and architectural arrangements. The information 
presented here in narrative form is based on data from the 848 schools participating in Phase 1 . 
Tables detailing the percentage of schools using each of these practices, usually by school level 
and location, are shown in Appendix H. Characteristics of school disciplinary practices were 
described by principals in the Phase 2 Principal Questionnaire (completed in Spring, 1998). 
Results are based on data from the 636 schools for providing information in Phase 2. Tables are 
shown in Appendix H. 4 

Provision of information. Between 78% and 92% of schools at all levels report providing 
isolated information about alcohol, tobacco, or other drugs. The detailed percentages are shown 
in Appendix Table H3.1 . Although research has usually failed to produce evidence of 
effectiveness of the isolated provision of information, the high percentages of use of this 
approach are not surprising given the obvious permeation of schools and the media with anti- 
drug messages. Smaller percentages of schools provide information about violence (62%) or 
accidents (56%). Information about risky sexual behavior is provided by 30% of elementary 
schools, 70% of middle schools, and 79% of high schools. 

Reorganization of grades, classes, or school schedules. Principals reported using a number 
of organizational arrangements to prevent problem behavior or promote school orderliness. 
Education researchers sometimes call these arrangements “school organization” characteristics. 
Table H3.2 shows which are often employed with the intent of preventing problem behavior and 
which are less often employed. 



4 Appendix tables show confidence intervals or standard errors that take the complex sample 
design into account. Readers should place more dependence on the standard errors or confidence 
intervals reported in the tables than on significance levels. 
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The most frequently used school organizational arrangement is what educators call 
“heterogeneous grouping,” that is, placing students who differ in conduct or ability together. 
Heterogeneous grouping, which 69% of schools report using to prevent behavior problems, is 
viewed by many educators and educational researchers as desirable because it avoids putting all 
difficult to manage or educate students together in groups, and it allows low achieving or 
behavior problem students access to their faster learning peers and more orderly classrooms. 
Despite this, 30% of schools report that they do group by ability and 13% do group by effort or 
conduct with the intention of preventing problem behavior. Some evidence implies that more 
(rather than less) problem behavior may occur when youths displaying relatively high amounts of 
problem behavior are grouped together (Dishion, McCord, & Poulin, 1999). 

Particularly in middle schools, principals report extensive use of “houses” or “teams” which 
generally means that a group of teachers is expected to have more familiarity and contact with a 
subset of students in the school. The use of a school-within-a-school - also more common in 
middle schools than in schools at other levels - is also an arrangement intended to provide 
smaller, more intimate, environments. Such arrangements may reduce problem behavior (D. 
Gottfredson et al., in press). 

Incongruously, 30% of principals report the use of stringent criteria for grade-to-grade 
promotion and 13% report relaxed promotion criteria as a way of reducing problem behavior. 
Among high schools, 43% of principals indicate that stringent promotion criteria are used to 
reduce or prevent problem behavior. Previous research implies that considerable dropout occurs 
in the early high school years, and that the behavior of the students who remain in school tends to 
be better than those who leave (G. Gottfredson, 1981). In middle school grades, where dropout 
is usually technically illegal, relaxing promotion criteria, which is reported by 26% of these 
schools’ principals, may be a way of promoting students who display problem behavior on to a 
high school. Fewer high school principals (8%) report relaxing promotion criteria. 

Nearly a third of principals indicate that they decrease class size as a way of reducing 
problem behavior. This suggests that problem behavior is costly, because small class sizes mean 
more classrooms and more teachers. Although not common, some schools segregate students by 
sex to reduce problem behavior. 

Altering school composition. One way for a school to avoid problem behavior is to avoid 
having students who are likely to engage in it. Conversely, one likely way to increase the level of 
problem behavior in a school is to concentrate youths whose behavior has proven troublesome in 
that school. In the Phase 1 principal questionnaire we asked principals to indicate which of 
several activities or arrangements influence who attends their school. Some of these 
arrangements would tend to attract academically-oriented students or students with good 
behavior, and others would tend to attract students who have displayed problem behavior in the 
past. Table 3.4 shows the percentage of schools employing each of 1 1 practices that influence 
student body composition. The most common practice cited by principals is, not surprisingly, 
attempting to have attractive educational programs, which was cited by 27% of principals. 
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Table 3.4 



Percentage of Schools Using Each of Several Activities or Arrangements That Influence 
Student Population 



Practice 


% 


95% Cl 


n 


Specialization in attractive educational programs such as 
science, music, technology c 


27 


24.0 - 31.0 


833 


Assignment of students with academic or learning problems to 
this school 


23 


20.0 - 26.8 


837 


Assignment of students with educational or behavioral 
problems to other schools 


22 


19.1 - 25.6 


835 


Admission fees or tuition 


21 


17.9 - 24.6 


837 


Assignment of students with behavior or adjustment problems 
to this school 


19 


16.3 - 22.6 


837 


Student recruitment programs 


14 


11.5 - 16.9 


839 


Selective admissions practices (e.g., high test scores, good 
conduct, high grade average, or other entry requirements) 


14 


11.5 - 16.6 


836 


Preference for students of a particular religion, faith, culture, 
ethnicity, or political inclination 


12 


9.4 - 15.0 


841 


Scholarships or tuition waivers 


12 


9.4 - 14.8 


839 


Assignment of students under court or juvenile services 
supervision to this school 


10 


7.7 - 11.8 


834 


Another practice or arrangement that influences the 
composition of the school’s student population 


11 


8.6 - 13.5 


823 



Note. % = weighted percentage; 95% Cl = 95% confidence interval; n = unweighted number of 
respondents. 



Almost equal percentages of principals report that problem students are referred to the school and 
from the school to other schools (19% to 23% of principals report these practices). 

A fifth (21%) of schools charge admission fees or tuition, and this practice is much less 
common in middle schools (8%) than among elementary (20%) or high (32%) schools. Details 
shown in Appendix Table H3.3 reveal that middle schools less often use student recruitment or 
selective admissions practices than do schools at other levels. 

To assess the extent to which schools are selective in their recruitment or admission of 
students or to which they are repositories for problem children, we composed two scales from 



items in our first principal questionnaire. A Selectivity scale is based on reports that schools 
actively recruit students, have selective admissions practices, prefer students of particular 
religion or other characteristics, have admission fees or tuition, or make use of scholarships or 
tuition wavers. A Problem Student Magnet scale is based on reports that students with behavior 
or adjustment problems are assigned to the school, students under court or juvenile services 
supervision are assigned to the school, or students with academic or learning problems are 
assigned to the school. The scores are expressed as T-scores (mean = 50 and standard deviation 
= 1 0 for schools), and detailed information about the distribution of selectivity is displayed in the 
top panel of Appendix Table H3.4. Urban and suburban high schools earn high scores on this 
index on average and tend to have a high standard deviation on the index. In other words, it is 
relatively common for urban and suburban high schools to attempt to influence the composition 
of their student membership by engaging in selective practices, but there is considerable 
variability in this practice among such schools. ' 

Selectivity is not a win-win proposition for schools. Schools that are unable to be selective 
or that do not attempt to be selective may tend to develop student populations who engage in 
higher levels of problem behavior. Variability among schools, particularly high schools, in 
selectivity may help to explain some of the variability in school disorder. Appendix Table H3.4 
does not reveal a particularly steep gradient by level or location for the Problem Student Magnet 
scale, however. 

Treatment or prevention interventions for administration, faculty or staff. A moderately 
large percentage of schools seek to prevent problem behavior and promote a safe environment by 
providing treatment or prevention services for administrators, faculty, or staff. Appendix Table 
H3.5 shows estimates that alcohol, tobacco, or other drug treatment or prevention services are 
provided by 59%, anger management or self control training by 51%, and other health or mental 
health services by 62% of urban middle schools. 

Architectural features. A class of arrangements that involve architectural or structural 
features of the school are also involved in school efforts to promote safety and reduce problem 
behavior. Table 3.5 shows that food service facilities dominate this category. (Details are shown 
in Appendix Table H3.6 for practices that may differ by both level and location.) This is not 
surprising, because during lunch periods large numbers of youths are apt to congregate in a single 
area and seek food at the same time. Kenney & Watson (1996) have described an intervention in 
which multiple lunch lines were put in place to reduce conflict in a single line. Among urban 
elementary schools, 54% use gates, fences, walls or barricades outside the building to promote 
safety or prevent problem behavior. In contrast, 25% of rural middle schools and 27% of rural 
high schools use gates, fences, walls, or barricades (see Appendix Table H3.6). Secondary 
schools, in particular, sometimes close or block off sections of the school building; 21% of 
middle schools and 28% of high schools engage in this practice. 
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Table 3.5 

Percentage of Schools Using Architectural Design or Structural Features to Prevent Problem Behavior or Promote School 
Orderliness, by School Level 
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School Discipline 

In the Phase 2 Principal questionnaire, we asked for reports about school rules, policies, 
regulations, laws, and enforcement. We asked about these activities in considerably more detail 
than we asked about other school-level activities because prior research (G. Gottffedson and 
Gottfredson, 1985; G. Gottffedson, 1984/1999) indicates that clarity of school rules and 
consistency in their enforcement is related to the level of school disorder. Also, national media 
attention has focused recently on certain school policies and practices thought by some observers 
to be effective for reducing drugs, violence and disorder. These include uniforms (Wingert, 
1999), metal detectors (Aleem & Moles, 1993), drug searches (Davis & Wilgoren, 1998), and so- 
called “zero tolerance” policies (Associated Press, 1999; Breckenridge, 1998; Churchill, 1998; 
Gabor, 1995). There is little or no useful research on the extent or usefulness of these practices. 5 

Less media attention has focused on some of the more routine or mundane things schools do 
to regulate student behavior, such as recognizing or praising students for desirable behavior or 
using ordinary social controls — often minor forms of punishment - to discourage misconduct. In 
this section, we first review information about formal school rules, regulations, and responses to 
student conduct. Then we review information about ordinary social responses to student 
conduct. 



School Rules 

Nearly all schools have formal written rules or policies about the time for student arrival at 
school, drugs, and weapons, as Table 3.6 shows. In addition, 75% or more of the schools have 
such written policies related to dress, visitor sign-in, students leaving campus, and hall 
wandering or class-cutting. Dress codes and rules about student mobility are less common at the 
elementary school level. Rules about carrying items or wearing clothing in which drugs or 
weapons could be concealed are more common at the middle school level. Visitor sign-ow? is a 
written policy far less often than visitor sign-in, and the requirement that visitors sign out is less 
common in high schools and is more common in suburban areas (Appendix Table H3.7). 
Twenty-six percent (26%) of the nation’s schools report having formal written policies about 
uniforms, but uniform policies are found in a much smaller percentage of rural schools than in 
urban and suburban schools. For example, 48% of urban elementary but only 8% of rural 



5 A report by Murray (1997) purports to assess the impact of school uniforms on school climate. 
The report provides no useful information, however, because it simply compares one school 
requiring uniforms and one not requiring uniforms that also differ in many other ways. For 
example they also have different principals, different counselors, etc. The largest difference 
between the two schools (ES = .39) was for a scale which contains items such as “teachers or 
counselors help students with personal problems” and “teachers and counselors help students 
plan for future classes and for future jobs.” If differences between the two schools were to be 
interpreted as effects of uniforms, it is not clear why uniforms would have their largest effect on 
students’ ratings of counseling. 
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Table 3.6 

Percentage of Schools with Formal Written Rules or Policies About Discipline by School Level 



OO 

cn 



co 

co 

— ^9 

Cd lO 

o o 

e2 'o 



d" 

ON 



.SP?i 

sc - 



u 



‘2 

£ 



>>°s 

i-4 *— ■ 

cd (N 

4-» , 

£ vo 
2 © 
S <N 

J2 II 

PQ r 



U 



3 

O 

X> 

cd 

So 

o 

o 

& 

a 

<D 



cd 



O 

tL( 



ON 


OO 


r- 


ON 


OO 


d- 


ON 


d- 


OO 


© 


Os 


ON 


ON 


OO 


OO 


oo 






d“ 


co 


NO 


CN 


,_li 


CN 


JL 


NO 


© 


d- 


oo 




ON 


ON 


ON 


OO 


oo 


r- 


r- 


NO 


CO 


CN 


OO 


tO 


d" 


NO 


tO 


o 


to 


ON 


CO 


NO 


ON 


ON 


ON 


OO 


OO 


oo 


r*- 


NO 


d" 


CN 



ON 


o 


r- 


NO 


uo 


r- 


oo 


CO 


o 


r- 


Os 


o 


ON 


Os 


oo 


ON 


os 


NO 


NO 


CN 


d- 




OO 


Os 


CN 


ON 


© 


oo 


to 


d- 


ON 


tO 

ON 


oo 


OO 


r- 


oo 


ON 


d- 


CO 




r**- 


OO 


CO 


CO 


00 


d - 


to 


uo 


CN 


, 


ON 


ON 


On 


On 




ON 


ON 


uo 


d- 


CN 





>— i 


ON 


O 


o 


to 


d" 


r- 


o 




ON 


OO 


CN 


U 


ON 


o 


o 


ON 


ON 


ON 


o 


OO 


NO 


CN 


CN 


Vp 

o x 

to 

ON 


to 


1 


1 


l 

OO 


l 

CN 




1 




NO 


1 

d" 


CN 

i 

CO 

CN 


ON 


oo 

ON 


NO 

ON 


oo 


OO 


ON 


00 

OO 


NO 


to 




II 


N® 


OO 


O 


oo 


CN 


oo 


to 


to 


d- 


CN 




si 


o x 


ON 


o 


ON 


ON 


oo 


ON 


ON 


r- 


NO 


CN 



Os 

Os 



tO 

Os 



oo 

ON 



O 

o 

X 

o 

00 

td 

*cd 

> 

t 

cd 

4 —* 

£ 

<D 

no 

£ 



<2 

<L> 

s 

H 



o oo r- 

ON ON OO 



OO ON NO 
oo oo 



(N CN 

ON ON OO 



ON 



00 



c 

.2 

cd 

i-H 

4 —* 

C/3 

*5) 

<L> 

-o 

c 



CO 

00 

2 



<D 

a o 

§ o 

CL c/a 
cd co 



co ^ 

& 4> 2 
Q £ Q 



U4 

O 

■ 4 — » 

"co 

> 



OO 






00 


0 


SO 










NO 


00 


d- 




to 






d- 


r^ 


CN 




NO 






to 


NO 


CO 










CN 


d- 


ON 




r- 






NO 


r- 


CO 










CJ 






0 




4— » 




0 J> 








C /3 

3 


cd 




£ 








r 




*£ 




0 


Id 


Oh 


op 




£ 




0 D CO 


<D 

r ■i 


£ 


d 




O 




£ Sp 




Lh 

cd 

0 


v — ✓ 

CO 

S-I 




CO 

CO 

JS 




2 

Cd ^7-4 
<D ^ 
? X 
^ 0 


O 

0 


<D 

j£ 


£ 

O 








0 

X 


4 -* 

00 


X 




5 — 1 

O 


x> 


t-H 

O x 


2 


£ 


’o 




00 




CO ^ 


3 


’> 

cd 


0 

X 

0 

CO 


03 


£ 

*£ 

<1> 

-O 


0 

£ 

op 


£ c 

q> — 
.1£ 00 


0 

0 

CO 

£ 


CO 


OX) 


r — s 


£ 


CO 


Cd) .2 


O 


£ 

<L> 


'£ 
’£ 
>— ( 


X 

0 

c 


cd 

£ 


L- 

O 

<4^ 


. G ^ 
*>s O 


& 

<D 


£ 


I-J 

T 3 


£ 


Id 






£ 


55 






sc 


> 


0 





tn 

co 

i 

CO 

CN 



OO 

CN 



C/3 

4 —* 

£ 

CD 

Td 

£ 

o 

&. 

oo 

<u 

5 -h 

4h 

0 

5 -h 

<D 

X) 

2 

£ 

£ 

~o 

<u 

4 —* 

X 

op 

*5 

1 
£ 

II 

K 

*cd 

> 

J-h 

<D 

4 —* 

£ 

<D 

O 

£ 

<D 

T3 

X 

£ 

O 

o 

no 

o x 

U~> 

ON 



O 

V© 

0 s " 

t/O 

ON 



da 



^ . 



o 

ERIC 



3-16 



r- 

03 



Elementary differs from middle and high,/? < .01. 
High differs from elementary and middle,/? < .001. 
Middle differs from elementary and high,/? < .001 . 



elementary schools report having uniform policies. Finally, the number of these written rules or 
policies also varies by school level (refer to Appendix Table H3.8), with middle schools 
reporting the more kinds of written rules than elementary or high schools. 

It is important not only to have clear rules about student behavior, but also to communicate 
these rules to all relevant parties. The vast majority of schools report distributing printed copies 
of the schools’ disciplinary policy to teachers (99%), students (96%) and parents (96%). Parents 
of high school students are less often provided with printed discipline policies, as are rural 
parents (see Appendix Table H3.9). The main exception to nearly universal distribution of 
printed policies is the 13% of high schools who report not providing parents with printed copies 
of school discipline policies in the current year. 

The phase 2 principal questionnaire asked for information about the current use or 
development of a variety of sound disciplinary procedures or practices. These included the 
maintenance of records, communication of rules or consequences, use of printed forms or other 
mechanisms for identifying and recording rule violations, use of specific methods for 
documenting due process, a system for investigating student circumstances, active specification 
of consequences for behavior, active development or modification of a discipline code, and 
student involvement in discipline. The majority of schools report the use of most of these 
procedures or practices. For example, 92% of schools report maintaining records of student 
conduct using forms, files, or computers. And, 72% of principals report that their discipline 
policies are under active development. The only practice about which we inquired that is not 
used by the majority of schools in the active involvement of students in the development of 
school discipline policies and procedures — reported by 46% of schools. Elementary schools less 
often report involving students in the development or modification of school rules, rewards, or 
punishments than do middle and high schools. Details are presented in Appendix Table H3.10. 
Notable differences in practices are not generally observed across school location, but rural 
schools less often reported using forms or other systems for identifying and recording rule 
violations when they occurred. 

Responses to Student Behavior 

Desirable behavior. Although some educators focus on rules and responses to misconduct 
when thinking about establishing and maintaining school safety and reducing problem behavior, 
it is generally also useful to consider arrangements or practices that tend to increase desirable 
behavior. Accordingly, the Phase 2 principal questionnaire asked for reports about the use of a 
range of potential responses to desired student conduct. Table 3.7 shows that the vast majority of 
schools - 81% to 96% - report the use of most of the social, activity, and materials reinforcers 
about which the questionnaire inquired. Many (61%) also reported using token reinforcers, 
which are coupons, tokens, or scrip that can be redeemed for backup reinforcers. (Appendix 
Table H3.1 1 shows details about the percentage of schools at different levels using each of a 
variety of responses to desirable student conduct.) The use of most types of positive reinforcers 
for desirable behavior is considerably less common at the senior high level. For example, 93% 
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of elementary schools report use of activity reinforcers (access to games, free time, library, 
playground) compared to 83% of middle schools and 64% of high schools. Only 8% of schools 
use money as a reward, although 1 8% of middle schools report the use of this reinforcer. 

Table 3.7 



Percentage of_ Schools Using Specific Responses to Desirable^ Student^ Conduct 



Response 


% 


95% Cl 


Informal recognition or praise (e.g., happy faces, oral praise, hugs) 


96 


94-97 


Formal recognition or praise (e.g., certificates, awards, postcard to the 
home, non-redeemable tokens) 


95 


92-97 


Job or privilege reinforcers (e.g., allowing student to erase chalk board, 
help the teacher, decorate a class) 


87 


85-90 


Activity reinforcers (e.g., access to games, free time, library, playground) 


84 


81-87 


Social rewards (e.g., lunch with a teacher, parties, trips with faculty) 


82 


78-85 


Material rewards (e.g., food, toys, supplies, etc.) 


81 


77-85 


Redeemable token reinforcers (e.g., coupons, tokens, or paper "money") 


61 


56-65 


Other response to desirable behaviors 


42 


33-51 


Money 


8 


6-11 



Note. Unweighted number of respondents ranges from 624 to 626. 95% Cl = 95% confidence 
interval for percentage. 



Undesirable behavior. Schools also employ a variety of responses to undesirable student 
conduct, and percentages are reported in Table 3.8. The most commonly reported responses to 
misconduct are mild forms of social control such as notifying parents (100%), talking to the 
student (100%), conference with parents (100%), oral reprimand (99%), brief exclusion from 
class (94%), and short-term withdrawal of a privilege (93%). More punitive responses such as 
suspension from school (reportedly used by 89%), restitution (86%), after-school detention 
(72%), and work assignments (70%) are also very common. Among the least common responses 
schools make are corporal punishment (17%) and Saturday detention (25%). Appendix Table 
H3.12 shows detailed results. 

The use of most kinds of responses tends to be reported more often in middle schools, most 
likely as a response to the higher level of discipline problems observed there. For example, the 
long-term (more than 5 days) withdrawal of a privilege (e.g., riding the bus, playground access, 
participation in athletics, use of the library) is reported by 57% of elementary schools, 91% of 
middle schools, and 80% of high schools. 

Some approaches to discipline about which there appears to be current interest among 
educators and delinquency prevention professionals are used by relatively few schools. Peer 



O 

ERLC 



3-18 



100 



mediation was reportedly used by 51% of schools, community service by 46%, and student court 
by 6% of schools. 6 

Table 3.8 



Percentage of Schools Using Specific Response^ to Undesirable Student Conduct_ 



Response 


% 


95% Cl 


Notifying parents about student’s behavior 


100 


- 


Conference with a student 


100 


- 


Conferences with student’s parents/guardians 


100 


- 


Oral reprimand 


99 


98-100 


Brief exclusion of students from attendance in regular classes (e.g. in-school 
suspension, cooling off room) 


94 


92-96 


Short-term (5 days or less) withdrawal of a privilege (e.g., riding the bus, 
playground access, participation in athletics, use of the library) 


93 


90-95 


Suspension from school (the exclusion of students from membership for 
periods of 30 days or less) 


89 


86-93 


Restitution (requiring a student to repay the school or a victim for damages or 
harm done) 


86 


82-89 


Sending student to school counselor 


85 


81-89 


Written reprimand 


81 


77-85 


Probation (a trial period in which a student is given an opportunity to 
demonstrate improved behavior) 


75 


71-80 


Calling or notifying the police 


74 


70-79 


Brief exclusion from school not officially designated suspension (e.g., sending 
students home with permission to return only with a parent) 


74 


70-78 


After-school detention 


72 


67-77 


Work duties, chores, or tasks as punishment 


70 


66-74 


Long-term (more than 5 days) withdrawal of a privilege (e.g., riding the bus, 
playground access, participation in athletics, use of the library) 


67 


62-72 


Writing assignments as punishment 


62 


58-67 


Transfer to one or more different classes within the school 


61 


57-66 


Expulsion from school (the exclusion of students from membership for periods 
of time over 30 days) 


57 


53-62 


Peer mediation 


51 


46-56 



continued . . . 



6 In the phase one principal survey even smaller percentages of schools reported prevention 
activities involving youth regulation of misconduct. Different questions produce different 
estimates, but they nevertheless converge in implying that these approaches are not used as 
widely as are others. 
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Table 3.8 (continued) 

Percentage^ oj_ Schools Usings Specific Responses^ to Undesirable^ Student Conduct 



Response 


% 


95% Cl 


Charging student with a crime 


51 


46-55 


Court action against student or parent 


48 


43-52 


Community service 


46 


41-50 


Mandatory participation of student in a special program 


44 


39-48 


Transfer to another school 


37 


33-42 


Saturday detention 


25 


21-28 


Other method of removal of students displaying problem behavior from the 
school 


24 


20-28 


Corporal punishment (e.g., paddling, spanking, striking) 


17 


13-20 


Mandatory participation of parent in a special program 


15 


11-18 


Other response to misbehavior 


10 


7-13 


Student court 


6 


4-8 


Informal physical responses (administration of discomfort through rubbing, 
squeezing, pulling, or the like) 


2 


1-3 


Note. Unweighted number of respondents ranges from 622 to 632. 95% Cl = 


95% confidence 



interval for percentage. 



In general, more severe responses (e.g., expulsion from school, Saturday detention, and 
calling the police) are used more often in secondary than elementary schools. Corporal 
punishment is reported much more often in rural (27%) than in suburban (6%) or urban (9%) 
schools. It is used least in Catholic schools and most in private schools. 7 

Suspension and expulsion. Schools suspend or expel students for misconduct ranging from 
truancy to possession of a weapon. For each of a range of offenses, principals were asked to 
indicate if they suspend or expel students automatically, usually after a hearing, or not usually. 
Results are displayed in Figure 3.1 (detailed tabulations are contained in Appendix Table H3.13). 
Schools are very likely to suspend or expel a student for possession of a gun, drugs, alcohol, or a 
knife. Suspension or expulsion occurs automatically or usually (after a hearing) in 91% or more 
of schools in response to these offenses. Suspension or expulsion for physical fighting, 
possession of tobacco, and use of profane or abusive language are also common, but are not 
usually “automatic.” 



’Although the sample contained only small numbers of Catholic (n = 46) and private (n = 50) 
schools that provided information on corporal punishment, private schools used more corporal 
punishment than public (p < .04) and Catholic (p < .001) schools. No Catholic school reported 
the use of corporal punishment. Among the 35 responding private high, vocational or 
comprehensive schools, 1 5 (unweighted) reported the use of corporal punishment. 
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Possession of a gun 
Possession of other drugs 
Possesson of alcohol 
Possession of a knife 
Physical fighting 
Possession of tobacco 
Profane or abusive language 
Chronic truancy 




0 % 20 % 40 % 60 % 80 % 100 % 



Automatic 



Usually after hearing 



1 1 Not usually 



LJ 



Figure 3.1 School Use of Suspension or Expulsion in Response to Specific Behaviors 



As with most disciplinary responses, use of suspension or expulsion tends to be reported by 
larger percentages of middle schools than elementary or high schools. But Figure 3.2 shows that 
while secondary schools report responding to fighting and chronic truancy with suspension or 
expulsion more than do elementary schools, they report responding with suspension or expulsion 
to the possession of tobacco less than do elementary schools. 

The large percentage of schools reporting the “automatic” suspension or expulsion of 
students is surprising. United States Supreme Court decisions in Wood v. Strickland (1975) and 
Goss v. Lopez (1975) imply that some degree of due process is required even for short-term out- 
of-school suspensions. Hearings for brief suspensions need not be elaborate or formal, but 
students must be notified of what they are accused of having done, told what evidence or 
information led the administrator to determine that the student violated a school rule, and be 
given an opportunity to respond. In the case of suspensions for over 10 days or of expulsions, 
hearings must be more formal. Written, specific, and timely notice of the charges and of a 
hearing were found to be required by the Supreme Court of Kansas in Smith v. Miller (1973). 

The Supreme Court of Montana also found that the charges must be specific in Board of Trustees 
of Billings School District No. 2 of Yellowstone County v. State of Montana (1979). In these 
more formal hearings, a student has a limited right to confront or cross-examine witnesses, 
according to the U. S. District Court for Arkansas in Dillon v. Pulaski County Special School 
District (1978). 
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Possession of tobacco 




Physical fighting 



Chronic truancy 




0% 20% 40% 

| | Elementary jjBBjl 



60% 



Middle 



80% 

High 



100% 



Figure 3.2 Percentage of Schools Reporting Use of Suspension or Expulsion for Specific 
Behaviors, by School Level 



Further clarification of what responding principals mean by “automatic” suspension or 
expulsion would be helpful. Response options presented to respondents in the principal survey 
included “usually after hearing,” which would suggest that when the “automatic” option was 
selected respondents are indicating that the response occurs without a hearing. Many school 
districts now post student handbooks or district policies on a Web site, so it is possible to 
examine what these documents say about suspension or expulsion for various offenses. These 
documents (e.g.. New Lebanon Middle School, 1999; Tremont Community Schools District 702, 
1999) usually do seem to call for appropriate levels of due process. They indicate that 
suspension or expulsion may result from violation of certain rules, and they spell out due process 
procedures for suspension or expulsion. It is possible to find evidence of more casual approaches 
to suspension in examining school handbooks. The Mount View Middle School (1997) student 
handbook makes no mention of a hearing. It states, “If the principal determines that a student is 
in possession of a weapon, the principal will secure the weapon, suspend the student, notify the 
respective Director and notify the police. . . . The student will be referred by the principal to the 
Superintendent for expulsion from the Howard County School System.” In contrast the County 
Discipline Policy (Howard County Public Schools, 1999) states, “Disciplinary action will be 
taken ... in accordance with Policy 3431, Discipline. Students who violate this policy may be 
suspended or expelled.” Policy 343 1 contains the usual Goss v. Lopez prescription for informal 
and expedited due process. 
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The Supreme Court’s opinion in Goss v. Lopez (1975) makes it clear that a hearing is 
required, that the hearing need not be elaborate, and that it should not be delayed. 8 “Due process 
requires, in connection with a suspension of 10 days or less, that the student be given oral or 
written notice of the charges against him and, if he denies them, an explanation of the evidence 
the authorities have and an opportunity to present his side of the story. The [Due Process] Clause 
requires at least these rudimentary precautions against unfair or mistaken findings of misconduct 
and arbitrary exclusion from school” (p. 582) 

It is evident from the large percentages of principal reporting “automatic” suspension or 
expulsion, rather than “usually after a hearing” that existing laws do not seem to be tying the 
hands of school administrators in removing students from school for a range of offenses. The 
evidence suggests that building-level administrators may treat due process requirements casually. 
In Goss v. Lopes the Supreme Court noted, “Students whose presence poses a continuing danger 
to persons or property or an ongoing threat of disrupting the academic process may be 
immediately removed from school. In such cases, the necessary notice and rudimentary hearing 
should follow as soon as practicable” (p. 582). It is hard to understand how possession of 
tobacco would pose such an ongoing threat that it would require suspension first and hearing 
later, yet two-thirds of schools indicate that suspension without hearing occurs for this offense. 

Discretionary Programs 

Certain kinds of prevention activity were the subject of more scrutiny than the school-wide 
arrangements and disciplinary practices examined so far. These are activities that tend to be 



8 The court was concerned with fundamental fairness: 

“The prospect of imposing elaborate hearing requirements in every suspension case is viewed 
with great concern, and many school authorities may well prefer the untrammeled power to act 
unilaterally, unhampered by rules about notice and hearing. But it would be a strange 
disciplinary system in an educational institution if no communication was sought by the 
disciplinarian with the student in an effort to inform him of his dereliction and to let him tell his 
side of the story in order to make sure that an injustice is not done.” (p. 580) 

“There need be no delay between the time “notice” is given and the time of the hearing. In 
the great majority of cases the disciplinarian may informally discuss the alleged misconduct with 
the student minutes after it has occurred. We hold only that, in being given an opportunity to 
explain his version of the facts at this discussion, the student first be told what he is accused of 
doing and what the basis of the accusation is” (p. 582) 

“In holding as we do, we do not believe that we have imposed procedures on school 
disciplinarians which are inappropriate in a classroom setting. Instead we have imposed 
requirements which are, if anything, less than a fair-minded school principal would impose upon 
himself in order to avoid unfair suspensions. . . . We stop short of construing the Due Process 
Clause to require, country wide, that hearings in connection with short suspensions must afford 
the student the opportunity to secure counsel, to confront and cross-examine witnesses 
supporting the charge, or to call his own witnesses to verify his version of the incident, (p. 583) 



more discrete and program-like. They are often considered “programs” by school personnel, and 
they may have names - although not all are named. These activities fall in fourteen of the 
categories or subcategories of the classification summarized in Table 3.2 and detailed in 
Appendix D. The “discretionary” activity types about which detailed information was sought 
are: (a) prevention curriculum, instruction, or training (including the use of cognitive-behavioral 
modeling methods of training or instruction); (b) behavioral or behavior modification 
interventions; (c) counseling, social work, psychological or therapeutic interventions; (d) 
individual attention interventions; (e) recreational, enrichment, and leisure activities; (f) 
interventions that change instructional methods or practices; (g) interventions that change 
classroom management methods or practices; (h) use of external personnel resources for 
classroom management and instruction; (i) interventions to change or maintain a distinctive 
culture or climate for interpersonal exchanges, or to communicate norms for behavior; (j) 
intergroup relations and interaction between the school and community or groups within the 
school; (k) formal youth roles in regulation and response to student conduct; (1) interventions that 
involve a school planning structure or process, or the management of change; (m) security and 
surveillance interventions, including efforts to exclude weapons and contraband; and (n) family 
interventions. 9 

Nature and Extent of “ Discretionary ” Programs 

The 14 kinds of “discretionary” prevention activities were the subject of greater scrutiny 
than other activities or arrangements described so far. Principals were asked to name up to five 
different program activities of each type that were currently underway and that were aimed at 
reducing problem behavior or creating a safe and orderly school environment. These reports 
(from the Phase 1 Activity Booklet accompanying the Phase 1 Principal Questionnaire for 
Program Identification), allow us to report not only the number of different types of 
“discretionary” prevention activities underway, but also how many different distinct activities of 
these types are in place. The information presented here is based on data from the 874 schools 
for which we had a response in Phase 1. Principals in these schools named 17,1 10 prevention 
activities. The next chapter of this report will describe the quality of these activities. 

On average, principals reported 9 of the 14 different types of discretionary prevention 
activities currently underway in their schools. Middle/junior high schools reported more types of 
activities than elementary or high schools, and rural schools reported fewer types of activities 
than suburban or urban schools. Means by level and location are shown in Appendix Table 
H3.14. 

The median number of different prevention activities named by principals within the 14 
discretionary types about which a detailed inquiry was made was 14. The distribution of the 



9 In Appendix D, list item (h) is a subcategory of interventions that change classroom 
management methods or practices, list item (k) falls under rules, policies, and regulations about 
discipline and their enforcement, and list item (a) occupies two categories. 
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number of activities named is positively skewed, with from 0 to 66 named per school. This 
median is the number of unique prevention activities named by the principal. So, for example, a 
principal may have listed a D.A.R.E. program under both the prevention curriculum and the use 
of external personnel resources categories. But D.A.R.E. is counted only once for the school. 
Appendix Table H3.15 details the median number of different prevention activities identified by 
school level and location. Middle/junior highs reported more activities (Mdn =16) than did high 
schools ( Mdn = 1 1); the elementary school Mdn = 14. Rural schools reported fewer activities 
(Mdn =11) than did urban schools (Mdn = 15); the suburban school Mdn =14. These figures do 
not count “additional” programs principals claimed, but which they did not name. For each 
program category in the Activity Detail Questionnaire, principals were asked to indicate how 
many additional programs they had if they ran out of spaces on the data collection form, which 
provided five spaces per category. Counting these claimed but unnamed activities added an 
average of four programs per school. Amazingly, one school reported 264 program activities 
when these unnamed activities are counted. 

The percentages of schools employing at least one activity in each of the 14 types of 
discretionary programs are shown in Table 3.9. Not surprisingly, the most popular type of 
discretionary prevention program in elementary schools entails prevention curriculum, 
instruction, or training. At the elementary level, 80% of schools report using a curricular or 
instructional approach to preventing problem behavior. The percentage is lower at the high 
school level, where 66% of schools report using such an approach. The average school uses 2.0 
distinct instructional or curricular activities to prevent or reduce problem behavior. Although the 
percentage of schools employing an instructional, approach is relatively high, we note that almost 
a quarter of schools (and almost a third of high schools) are not using this approach. Evidence 
implies that cognitive-behavioral social skills training can produce modest reductions in problem 
behavior (D. Gottfredson et al., in press), so there appears to be potential for broader application 
of effective approaches to preventing problem behavior. 

Counseling, social work, psychological, or therapeutic interventions are also very common, 
reported by 75% of schools. A somewhat higher percentage of middle schools (83%) reported 
using this form of intervention to prevent problem behavior than did elementary or high schools 
(each 74%), but the confidence intervals for these percentages overlap slightly. (Details of the 
percentages of schools reporting the use of each type of discretionary program are shown in 
Appendix Tables H3.16 and H3.17, along with confidence intervals for the percentages and the 
average numbers of activities reported.) There is a tendency for most discretionary program 
types to be represented in a larger percentage of middle schools than of schools at other levels. 
For example, 70% of middle schools but 65% of elementary and 57% of high schools use 
behavior modification or behavioral programming to prevent or reduce problem behavior, and 
62% of middle schools but 41% of elementary and 40% of high schools report involving youths 
in regulating and responding to student conduct. An exception to the observation that larger 
percentages of middle schools than other schools report use of activities is that the percentage of 
elementary schools reporting the use of prevention curricula and external personnel resources are 
higher than corresponding percentages for secondary schools - significantly higher than the 
percentages for high schools. This may be due to the more extensive use of classroom aides in 
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Table 3.9 



Percentage of Schools Using Each Type of Discretionary Prevention Activity, by School 
Level 



Type of prevention activity 


Elem. 

n=301 


Percentage for: 

Middle 

«=301 


High 
«= 272 


% 


Total 

/V=874 

Mean 

Number 


Prevention curriculum, instruction, or training 
(including the use of cognitive-behavioral 
modeling methods of instruction) 


80 


77 


66 


76 


2.0 


Counseling, social work, psychological or 
therapeutic interventions 


74 


83 


74 


75 


1.4 


Use of external personnel resources for L 
classroom management and instruction 


76 


73 


63 


72 


1.4 


Interventions to change or maintain a 
distinctive culture or climate for 
interpersonal exchanges, or to increase 
adherence to norms 


66 


74 


59 


66 


1.6 


Behavioral or behavior modification 
interventions 


65 


70 


57 


64 


1.2 


Recreational, enrichment, and leisure activities 


61 


73 


66 


64 


1.7 


Interventions that change instructional methods 
or practices 


64 


66 


54 


62 


1.3 


Individual attention interventions, e.g., 
mentoring/tutoring 


55 


64 


63 


58 


1.1 


Intergroup relations and interaction between the 
school and community or groups within the 
school 


56 


68 


54 


57 


1.5 


Interventions that change classroom 
management methods or practices 


59 


63 


51 


57 


1.0 


Interventions that involve a school planning 
structure or process, or the management of 
change 


57 


67 


52 


57 


1.1 


Security and surveillance interventions, 
including efforts to exclude weapons and 
contraband 


51 


66 


57 


55 


1.2 


Family interventions 


59 


60 


42 


55 


1.0 


Formal youth roles in regulating and 
responding to student conduct 


36 


55 


42 


40 


.6 



Note, ns are unweighted number of respondents. Table shows percentages reporting at least one 
activity for each type of activity. 
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elementary schools as well as the more frequent presence of Drug Abuse Resistance Education or 
other curricula in elementary schools. 

For some types of activities, smaller percentages of rural schools than of urban schools 
report using the activity (see Appendix Table H3.17). For example, a smaller percentage of rural 
schools than of urban schools report having mentoring programs to prevent or reduce problem 
behavior (50% versus 69%), activities to promote intergroup relations or interaction between the 
school and community (49% versus 66%), and security or surveillance programs (46% versus 
61%). 

Summary: Discretionary Programs. A very large percentage of the schools use each type of 
“discretionary” prevention activity. The percentages range from 40% for programs involving 
youth in the regulation of student conduct to 76% for prevention curricula. 

Multi-component and “Packaged” Programs 

Multi-component programs are those that include more than one type of prevention activity 
(e.g., a prevention curriculum in combination with activities to change school norms; or tutoring 
along with a behavior modification intervention). “Packaged” programs are “off-the-shelf’ or 
“canned” programs that are marketed to schools. Multi-component programs are of special 
interest because there are multiple “risk factors” or statistical predictors of problem behavior. 
Therefore there is reason to believe that multi-component programs may address causal factors 
more comprehensively than do interventions directed at single risk factors (see Conduct 
Problems Prevention Research Group, 1999a, 1999b). Packaged programs are of special interest 
because (a) they may be held out to consumers as products that are effective in reducing problem 
behavior, (b) development work may have gone into producing a product that is easy to 
implement, (c) they may make it easier for local implementers to apply standardized programs, or 
(d) they may be difficult to adapt to fit local conditions. It can be argued that without adaptation, 
the feasibility or appropriateness of canned programs may be limited. Conversely, it can be 
argued that adaptation may introduce changes that limit program effectiveness. Because they are 
of special interest, information about multi -component and packaged programs is described in 
this section. 

Multi-component programs. All told, principals named 17,1 10 prevention activities in the 
Activity Detail Questionnaires. Of these, 17% were multi-component programs. Reviews of 
school-based prevention programs (Elias et al., 1994; Hawkins et al., 1998) have suggested that 
programs targeting several risk factors for problem behavior and programs targeting several 
different domains of student life can be expected to be more potent. Of the nearly 3,000 (2,871) 
multi-component programs named in the present survey, 10 most (65%) combined only two 
different types of activities, but this number ranged up to seven. Certain types of activities tend 
to be “stand-alone” activities. For example, only 5% of security activities and 6% of recreation 



l0 A multi-component program is a named activity that was listed by the principal under more 
than one of our 14 discretionary activity types. 
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activities were also associated with another activity type. Other types tend to be part of a multi- 
component program: 41% of activities involving youths in the regulation of student conduct were 
also associated with another type of activity. Appendix Table H3.18 shows the percentage of all 
activities named in each category that were listed as an activity in at least one other category. 
Chapter 5 will examine the relative quality of activities implemented as part of multi-component 
programs compared with similar “stand-alone” activities. 

Packaged programs. An activity or program was regarded as “packaged” if it was 
mentioned by a large number of persons completing the activity booklet. Several easily 
recognized or trademarked programs were identified in this way. Table 3.10 shows the 1 1 
packaged programs identified in this manner and the percentage of schools whose principals 
reported using each of these programs. Note that the table lists standardized or structured 
programs, such as D.A.R.E., and G.R.E.A.T., as well as programs which may have relatively 
heterogeneous content - such as peer mediation and conflict resolution, because a variety of 
different packages with this designation are marketed by commercial vendors or by school 
districts. The most widely applied of these programs is clearly D.A.R.E., with 34% of all schools 
and 48% of elementary schools reporting its use. Peer mediation and conflict resolution 
programs are the second and third most widely used packaged programs adopted by schools to 
prevent or reduce problem behavior. The percentage of schools reporting the use of these 
packaged programs differs by school level: High schools are far less likely to make use of these 
“canned” programs than are elementary and middle schools. Only 37% of high schools 
compared to 65% and 67% of middle and elementary schools use these programs. The greater 
use of packaged programs in elementary schools is due largely to D.A.R.E. Middle/Junior high 
schools are more likely than others to use peer mediation, with 36% of middle schools compared 
to 1 1% of elementary and 13% of high schools reporting the use of peer mediation. 

The results summarized in Table 3.10 imply that most elementary and middle schools and 
many high schools use at least one packaged program, i.e., a program that was developed outside 
the school and marketed to it in some manner. Chapter 5 will contrast the quality of 
implementation for these packaged programs with “home grown”programs. 

In the Phase 1 Principal Questionnaire and Activity Detail Booklet, principals were asked to 
name prevention activities in each of 14 categories. The categories under which principals listed 
specific packaged programs provides some insight into how they view the programs operating in 
their schools. Different principals listed specific packaged programs in different categories. In 
addition, a principal sometimes listed a specific packaged program in multiple categories. Table 
3.11 shows how principals listed each of the 1 1 packaged programs. For example, 47% of the 
listings for Assertive Discipline were under the category “improvements to classroom 
organization and management” (which is, incidentally, how we would have classified it), and 
33% of the listings for Assertive Discipline were under the behavior management category 
(which also makes sense). Some principals listed this program under prevention curriculum, 
culture or climate change, or improvements to instructional practices. The observation that 9% 
listed Assertive Discipline under prevention curriculum suggests that either some principals do 
not have thorough information about what the program entails, or that their schools implement it 
in an unusual way. In general, the principal’s descriptions of the packaged programs in Table 
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Table 3.10 

Percentage of Schools Using Each Packaged Program, by School Level 
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Suburban 31 Suburban 

Rural 37 Rural 



Table 3.1:1 

Percentage of Listings of Packaged Programs in Each Category of Prevention Activity 
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all mentions of each packaged program that were in each prevention category. 



3.11 show convergence with the intended design of each program, despite a considerable amount 
of divergence or misclassification. Conflict resolution and peer mediation programs are 
classified in more heterogeneous ways than the other packaged programs. This suggests that 
these labels stand for different combinations of activities - possibly reflecting variability in 
content or process across the different “brands” of peer mediation and conflict resolution that are 
marketed or “disseminated” to schools. Heterogeneity implies that it may be difficult to 
accumulate meaningful information about the effectiveness of interventions such as conflict 
resolution and peer mediation without identifying program subtypes when research is conducted. 

Conclusions About Extent and Nature of Prevention Activity 

The typical school uses many activities and many different kinds of activities to prevent or 
reduce problem behaviors or promote a safe and orderly environment. Such extensive activity 
and breadth of coverage may be valuable, because having many different activities is likely to 
increase the number of risk or protective factors targeted. It is also possible, however, that by 
attempting so many different activities, schools spread their efforts too thin, diminishing the 
quality of each effort. Program quality is explored in the next chapter. 

Middle and junior high schools generally report more prevention activity underway than do 
elementary and high schools. This may reflect the higher level of problem behavior experienced 
in schools serving youths in middle grades. 

The broad range of different types of prevention policies, practices, arrangements, and 
activities used by schools to prevent problem behavior and promote a safe and orderly 
environment contrasts with some common perceptions about the nature of school-based 
prevention activities. Popular guides and lists of programs are most often dominated by 
curriculum packages (e.g.. Drug Strategies, 1998). And guides pertaining to school safety often 
focus on security arrangements or identifying troublemakers (e.g.. National School Safety Center, 
1998; Stephens, 1995). While prevention curricula are widely used in schools, schools are 
actually using a wide variety of different strategies to try to reduce problem behavior. The degree 
of effectiveness of most of these activities is unknown. 

Some of the strategies schools use to reduce problem behavior and increase safety and 
orderliness are relatively inexpensive and easy to accomplish (e.g, using heterogeneous grouping, 
or distributing information, creating grade level houses or teams), while others are costly and 
difficult to implement (e.g., decreasing class size, employing stringent grade-to-grade promotion 
standards). Different schools employ different strategies. At present, there is a limited base of 
dependable information to guide schools in selecting approaches to the prevention of problem 
behavior. Despite the availability of multiple evaluations of some instructional packages, there is 
a shortage of useful evaluations of changes in class size or promotion practices on problem 
behavior. Useful evaluations are lacking for most practices employed by schools to promote a 
safe and orderly environment and to prevent problem behavior. It should be possible, however, 
to capitalize on the large amount of natural variation in these practices to learn more about their 
potential to reduce problem behavior. 
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Schools make substantial use of architectural and structural arrangements to prevent problem 
behavior or promote school safety. Routine activity theory (Cohen & Felson, 1979; Felson & 
Cohen, 1980) suggests that manipulating these features may reduce school crime by reducing 
opportunities for offenders and victims or targets of crime to come together in time and space. 
Schools use strategies that can be interpreted in the context of routine activity theory or the 
situational crime prevention perspective as an “opportunity blocking” approach (Clarke, 1995; 
Eck, 1997). Urban schools are more likely to use gates, fences, walls, and barricades, and to 
physically block off sections of the building than are schools in other locations. Again, natural 
variation in the use of these architectural or structural arrangements could be exploited to learn 
more about their effects. 

Most schools report that they have strict rules about dangerous behaviors and the 
possession of weapons, communicate those rules, and apply severe consequences when these 
rules are broken. It is unlikely that extreme school violence (such as the highly publicized recent 
shootings in schools) occurs because of lax rules about carrying weapons in school. 

Most schools report that they have systems to keep track of individual student behavior, 
have a discipline referral system, communicate rules, have a systems for investigating 
infractions, and have procedures for achieving and documenting due process when they suspend 
students. Most principals report that their schools have written policies about behaviors they 
wish to prohibit, and principals report that these policies are communicated in writing to relevant 
parties. 

But schools often fall short, in using discipline practices that accord with.practices that , 
research has found to be associated with school safety. Principals report that their schools tend 
to rely on punitive responses to misbehavior more than on positive reinforcement of desirable 
behavior. They tend to make use of a narrower range of possible reinforcers for both negative 
and positive student behaviors than is potentially available. There is much room for 
improvement in the area of school discipline management, but recent calls to make rules for 
serious behavioral infractions stricter (e.g., Associated Press, 1999; Bush for President, 1999) 
may overlook other important areas where improvement is needed and possible. 

Finally, principals’ reports summarized in this chapter show that many “packaged” programs 
are being used in the nation’s schools, and that many programs are broad in scope (e.g., part of 
multi-component efforts). 

In the next chapter, we examine the quality of prevention programming in schools, 
comparing levels of strength and integrity in typical school programs with what has been shown 
in research to produce desirable outcomes. That chapter also explores the extent to which 
“packaged” and multi-component programs are implemented with as much strength and integrity 
as “home-grown” and stand-alone programs. 
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Program Intensity and Use of Best Practices 

The previous chapter reviewed evidence from the National Study of Delinquency Prevention 
in Schools about what schools do to prevent or reduce problem behavior and promote a safe and 
orderly environment. It revealed that schools undertake a great amount and a great variety of 
activity in pursuit of these aims. In this chapter we turn our attention to how well schools 
implement what they undertake - the quality of implementation. 

Importance of Intensity and Fidelity to Good Practices 

Most reviews of prevention practices and the growing number of lists of effective practices 
intended to guide prevention practitioners are organized according to type of preventive 
intervention. One example of an organization by type of preventive intervention is the 
classification that structured the present research (Appendix D). Practices or programs can also 
be ordered along dimensions of quality. Quality of implementation - the strength of intervention 
and fidelity to a useful plan for intervention - may be as important as the type of program. 

Until now, we have had little information about the quality of implementation of prevention 
programs in schools. Some information of this type comes from an evaluation of the Department 
of Education’s Safe and Drug Free Schools and Communities Program (Silvia & Thome, 1997), 
which found that programs implemented by schools are not nearly as comprehensive or extensive 
as the programs found to be effective in research. That study also found that program delivery at 
the school level is inconsistent: the amount and content of prevention programming varies greatly 
from classroom to classroom and school to school - even in districts trying to deliver consistent 
programs. Teachers often reported that they had not received sufficient training, were not 
comfortable with the subject matter or the teaching methods recommended in the curriculum 
materials, and many reported that teaching prevention-related material was of relatively low 
priority in an already full school day. 

We have only limited understanding of the effectiveness of research-based programs when 
they are implemented under more natural conditions, but as we noted in chapter one, those few 
studies that have measured the level of implementation show remarkable variation in the strength 
and integrity of implementation, and show that the strength of implementation is related to 
program outcomes. Botvin, Baker, Dusenbury, Tortu, and Botvin (1990) directly examined 
variability in the quality of implementation of the Life Skills Training (LST) program and the 
effects of this variability on program outcomes. Botvin’s team carefully measured the amount of 
the LST curriculum delivered after teacher training. The percentage of the materials covered in 
actual implementation varied widely from school to school - from 27% to 97%, with an average 
of 68%. Only 75% of the students were exposed to at least 60% of the program. Botvin et al. 
also showed that when the program is delivered poorly, positive effects are not found. In reports 
on the effectiveness of LST, Botvin and colleagues typically exclude those classrooms which 
delivered less than 60% of the program in summarizing outcomes. Although the findings of 
Botvin and his colleagues are most definitive, scattered evidence can be found in other published 
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literature (summarized in D. Gottfredson, Gottfredson, & Skroban, 1998; see also G. 

Gottfredson, Jones, & Gore, 1 999) that the quality of implementation matters. Positive results of 
prevention programs are found in studies and in sites within studies in which high 
implementation was achieved, but they are not generally found when implementation was poor. 

The view that quality of implementation is important 'and far from assured, only now 
emerging in the delinquency and drug prevention fields, has prevailed for some time in the 
broader study of educational innovation. In the 1970s and early 1980s, several studies of school 
innovation reported similar results: Attempts to improve educational practices in schools usually 
resulted in incomplete, inadequate, or sporadic implementation (Berman & McLaughlin, 1978; 

G. Gottfredson, Gottfredson, & Cook, 1983; Hall & Loucks, 1977; Sarason, 1971). Sarason 
characterized many educational innovations as “nonevents,” and Miles (1981) described some 
innovations as “ornaments.” In the broader educational arena, some emphasis has been placed on 
creating organizational arrangements and conditions to support higher quality implementation. 
Some of these strategies include the use of quality assurance teams, peer coaches, and master 
teacher arrangements that assign training and technical assistance roles to more experienced and 
skilled teachers. Some marketers of educational programs offer them only to schools where 
teachers vote overwhelmingly to adopt them in order to improve the prospects for 
implementation (Jones, Gottfredson, & Gottfredson, 1997; Mathews, 1999). 

It may be that the quality of implementation of prevention programs matters more than the 
type of prevention intervention. For example, a comprehensive instructional program may be 
effective for reducing problem behavior if it focuses on a range of social competencies (e.g., self- 
control, stress-management, responsible decision making, social problem solving, and 
communication), uses behavioral modeling principles, and is delivered over a long period of time 
to continually reinforce skills and provide ample practice. But, an instructional program may be 
ineffective for reducing problem behavior if it is brief, of low dosage, or lacks key content or 
instructional methods. In addition, categorical labels applied to prevention or treatment 
programs by meta-analysts or others who attempt to summarize results of program evaluations 
may mask large amounts of variability within category in the quality of implementation. 

The remainder of this chapter explores variability in the intensity and fidelity to good 
practices of the prevention activities examined in the National Study of Delinquency Prevention 
in Schools. 

Data and Measures for Examining Program Quality 

Data to describe the quality of prevention activities come from the reports of principals in 
the phase 2 questionnaire about school-wide activities and from activity coordinators in Activity 
Coordinator Questionnaires asking about the fourteen different types of “discretionary” program 
activity. Some explanation of the Activity Coordinator survey is required, and it is provided in 
the following paragraphs. 
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A total of 17,1 10 prevention activities were listed in the Activity Detail Booklets 
accompanying the Phase 1 Principal Questionnaire for Program Identification (obtained in the 
Spring of 1997) or a brief supplementary questionnaire for phase 1 nonrespondents 1 (obtained in 
the 1997-98 school year). Because some schools listed a large number of activities, we randomly 
sampled a maximum of one activity in each of the 14 categories in order to reduce the response 
burden on schools. In addition, if any D.A.R.E. or peer mediation program was not sampled 
randomly, it was added to the sample. This produced a sample of 8,043 prevention activities for 
which we set out to obtain detailed information in Activity Coordinator Questionnaires. 

As part of our preparations for the Spring 1998 school surveys, we telephoned schools to 
accomplish three things: (a) seek their participation in the phase 2 surveys, (b) verify the 
existence of sampled activities for which we intended to seek detailed reports, and (c) identify 
potential alternate respondents when a single individual had been nominated as an informant 
about multiple prevention activities. 2 Of the 8,043 activities, 796 (9.9%) were found not to exist 
at phase 2 survey time, and 127 activities (1.6%) were de-selected to avoid overburdening 
respondents. In addition, clerical errors led to sending a Activity Coordinator Questionnaire in 
the wrong category in 16 instances (0.2%). 3 Accordingly, there were 7,104 potential responses to 
Activity Coordinator Questionnaires (representing 88.3% of all activities initially sampled). In 
all, 3,691 completed questionnaires were obtained (45.9% of all activities initially sampled, and 



'For secondary schools that had failed to participate in the Phase 1 survey but had not 
affirmatively refused, and for which we were successful in obtaining school district approval to 
proceed with a survey, a one-page form was used to seek the identification of prevention 
activities in the same 14 areas covered by the regular Phase 1 questionnaires. For a small number 
of schools ( N = 44) this supplementary procedure was the source of identified prevention 
activities. 

2 In some cases the principal listed him or herself or one or two other persons as the individuals 
who could describe a number of activities. We wished to avoid requesting the same individual to 
describe more than two activities. Therefore we requested the names of other persons in the 
school who could describe some activities. When a principal insisted that only he/she (or only 
another individual) could describe a number of activities, activities were randomly subsampled 
so that no individual would be asked to complete more than two questionnaires in phase 2. For 
principals, one of these was the Phase 2 Principal Questionnaire. 

J We now recommend using different color paper or ink to help distinguish questionnaires that are 
otherwise similar in appearance. 
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51.9% of the 7,104 questionnaires delivered to respondents). 4 An accounting of sampling and 
response rates was provided in Table 1.8. 

Recall that of the 8,043 sampled prevention activities about which we inquired, we 
determined that 796 (about 10%) no longer existed by the time we asked activity coordinators to 
describe the programs several months later. This is undoubtedly a lower bound estimate of the 
percentage not in existence in the school year after the activities were initially identified. We 
obtained verification that 5,067 (63%) did exist at that time, but were unable to obtain an 
indication about the continuing existence of 2,180 activities (27% of the total). Activities in 
some categories were more likely to be found still in existence than other activities. A high 
percentage (92%) of counseling activities existed, whereas a smaller percentage (79%) of 
programs that involve youths in school discipline (e.g., peer mediation programs) were still in 
existence. Details are shown in Appendix Table B4. 1 . 

Measures of Quality of Discipline 

Two scales were created to measure the consistency of enforcement of school rules based on 
the reports of principals. The short Predictable Response scale is based on reports that 
disciplinary responses to specific infractions will be highly predictable, whereas the Conditional 
Response scale measures the extent to which discipline decisions are made by taking 
characteristics of a referring teacher or of a student into account. 5 High scores on the Predictable 
Response scale are desirable, according to previous research showing that clarity of school rules 
is related to lower levels of school disorder (G. Gottfredson & Gottfredson, 1985; G. 

Gottffedson, 1999). Low scores on the Conditional Response scale appear desirable, because 
research directed at improving the consistency of school discipline suggests that it is necessary to 
overcome disciplinarian’s tendency to condition responses on what teacher or kind of student is 
involved in order to increase consistency and fairness (D. Gottfredson, 1988; D. Gottfredson, 
Gottfredson, & Hybl, 1993). Appendix Table H4. 1, which reports T-scores by level and location 
for these and other scales, shows that average scores on the Conditional Response scale and 
Predictable Response scale are similar across level and location, although Predictability may tend 
to be a bit lower on average in elementary schools. 

Tables of mean T-scores convey a form of normative information, but they do not convey 
information that can be judged according to any criterion of adequacy. Accordingly, we formed 



4 Most of the non-response was due to the 285 schools which returned none of their coordinator 
surveys. Of the 554 responding schools, the percentage of coordinator surveys returned ranged 
from 7% to 100%, with an average of 83%. Survey response rate was not significantly related to 
either activity type or the overall quality of programs in the school. 

5 Item content of these scales is shown in Appendix E. 



another kind of composite measure intended to communicate information about the quality of 
school discipline practices. These composites are as follows: 6 

1 . Communication and Documentation. 7 items about the extent of distribution of the school’s 
discipline policy and current efforts to maintain or use procedures for documentation. Higher 
scores mean that a larger number of sound communication and documentation practices are 
employed. 

2. Range of Appropriate Responses to Misconduct. 17 items about a variety of potential 
responses to misconduct schools might exercise, ranging from brief exclusion from class, use 
of peer mediation or student court, detention, reprimands, and notifying parents, to 
community service. Higher scores mean that a greater variety of appropriate responses are 
employed. 

3. Range of Responses to Desirable Conduct. 7 items about the variety of potential responses to 
desirable student behavior that a school might exercise, ranging from material rewards, 
through informal recognition or praise, activity or privilege reinforcers, to formal recognition 
or praise. Higher scores mean that a greater variety of potential reinforcers are employed. 

4. Disciplinarian Consistency. 3 items about whether specific disciplinary responses are 
independent of the source of referral, identity of the decision maker, or the student 
disciplined. Higher scores imply greater consistency. 

5. Predictable Disciplinary Decision Making. 2 Likert-type items about whether students and 
teachers can predict the administration’s disciplinary response. Higher scores imply greater 
predictability. 

6. Adequacy Composite. The percentage of composites. 1 through 5 for a school that were 
above a designated cut point. A higher score means that more of the composites were above 
a minimum threshold. 

The six criteria are summarized in Table 4.1, which shows the potential range of scores, the 
observed range, and the cut point for “adequacy” adopted. 

Measures of Quality of Discretionary Activity or Programs 

Activity Coordinator Questionnaires were designed to gather information about the quality 
and quantity of services provided. When possible, the same questions were asked about each 
type of program or activity. Often, however, the wording of a question that worked for one 
program activity type was inappropriate for another program activity type. For example, 
questions about the number of lessons or sessions were more appropriate for curricula or 
counseling activities than for school planning or security activities. Questions were therefore 
tailored to each activity type while attempting to retain as much consistency in measurement 
content across questionnaires as possible. Descriptive data about the content and objectives of 
discretionary activities are presented in Appendix Tables H4.2-H4.17. 



6 The specific item content for each scale can be found in Appendix E, section 2. 
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Table 4.1 



Criteria Used to Judge Adequacy of School-Wide Disciplinary Practices 



Measure 


Potential range 
of scores 


Observed range 
of scores 


Cut point 


Communication and documentation 


0-100% 


14 - 100% 


70% 


Range of appropriate responses to 


0 - 100% 


12 - 94% 


70% 


misconduct 








Range of responses to desirable conduct 


0 - 100% 


0-100% 


70% 


Disciplinarian consistency 


0 - 100% 


0-100% 


70% 


Predictable disciplinary decision making a 


1-5 


1-5 


4 


Adequacy composite b 


0 - 100% 


0 - 100% 


70% 



a The average of two Likert-type items about how often students or teachers can predict the 
administration’s disciplinary response because they know the punishment for the offense. 4 = 
“most of the time,” 5 = “almost always.” This scale has a small (. 1 5) correlation with the 
measure of disciplinarian consistency. 

b The percentage of criteria above the cut point. This composite does not form a scale, with a 
only = .19 for 189 elementary schools and a = .24 for 380 secondary schools. 

Indicators of intensity included level of use by school personnel, frequency of operation, 
duration, number of sessions, frequency of student and staff participation, the ratio of providers 
to students in the school, and proportion of students involved in the activity. “Level of use” was 
viewed as a continuum (Hall & Loucks, 1977) ranging from no knowledge or awareness of an 
activity, through having acquired information or training, trying the activity, to using or applying 
regularly. Respondents indicated the level that characterized use of a practice in their schools. 
“Best practices” scales were scored by calculating the proportion of the identified research-based 
practices with respect to content or methods used in a particular activity or program. To develop 
these scales, research-based practices were identified for each program type independently by the 
two principal authors of this volume. Each author identified those practices that would be 
indicated by research about which he or she had knowledge. There was high agreement, and 
discrepancies in judgments were discussed and resolved by referring to the evidence. See 
Appendix E for the specific practices included in each best practice scale. 

The indicators of intensity and fidelity to good practice are shown in Table 4.2, together with 
the range of responses available for each. Each of these measures is examined in this chapter, 
along with an Intensity scale composed of three items available for most activity categories. 
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Table 4.2 

Summary of Measures of Intensity and Fidelity to Good Practice 



Intensity /Fidelity Measure 


Range of Responses 


Level of use by school personnel 


1 At least one person in the school knows 
something about it 

5 One or more persons is conducting 
activity on a regular basis 


Best practices: content 


0 to 1 (See note) 


Best practices: methods 


0 to 1 (See note) 


Frequency of operation 


1 Special occasions once or twice a year 
3 Continually throughout the year 


Number of lessons/sessions 3 


Write-in of exact number (natural log of the 
number is also examined due to positive skew 
in the distribution of the number) 


Duration 3 


1 Less than a day 
7 More than a full school year 


Frequency of participation - students 3 


1 Monthly or less often 

6 More than once per day 


Frequency of participation - staff 


1 Monthly or less often 

6 More than once per day 


Ratio of providers to students in the school 


1 00(ln(jVp/A/. + 1)), where N p = number of 
persons providing the service, and N s = the 
number of students in the school 


Proportion students exposed or participating 


Generally, NJN S , where N e = number of 
students exposed or participating, and jV s = 
number of students in the school. For the 
category “Youth Participation in School 
Decision Making,” N e = disciplinary 
incidents handled by student court or peer 
mediation, and N s = disciplinary incidents 
handled by student court, peer mediation, or 
the administration. 



Note. Scores for the “best practices” scales are the proportion of the identified best practices 
(content or methods) reportedly used in a particular activity or program. 

3 Included in composite Intensity scale 
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Quality of School-Wide Discipline 

Table 4.3 shows how the schools measure up on the summary indicators for quality of 
school-wide discipline. Principals in the nation’s schools generally claim to be communicating 
rules to teachers, parents and students and keeping track of student conduct. In all, 93% of 
schools are judged to have “adequate” communication and documentation, with 92% of 
elementary, 98% of middle and 94% of high schools exceeding the threshold for adequacy. The 
majority of schools fall short of our adequacy criterion in all of the other discipline areas: range 
of responses to misconduct, range of responses to desirable conduct, disciplinarian consistency, 
and predictable disciplinary decision making. Schools tend to use relatively small percentages of 
the possible responses available for misconduct and good conduct. Only 27% of schools use 
70% or more of the possible responses for misbehavior, and only 20% of schools use 70% or 
more of the possible responses to desirable behavior. The use of these responses differs 
according to school level: Only 7% of high schools meet the adequacy cut-point for positive 
responses; a larger percentage of middle (15%) and elementary schools (26%) meet the adequacy 
cut-point. Only 1 5% of elementary schools meet the 70% criterion for responses to negative 
behavior: higher percentages of middle (52%) and high (42%) schools meet the adequacy 
threshold. 

Research implies that consistency and predictability in disciplinary responses produce 
greater orderliness (G. Gottfredson & Gottfredson, 1985; D. Gottfredson, 1987; D. Gottfredson et 
al., 1993; G. Gottfredson, 1999), and consistency is commonly recommended as a sound 
disciplinary strategy (e.g., Goal 6 Work Group, 1993), yet fewer than half of our nation’s schools 
fall above the adequacy cut-point selected for the two relevant indicators based on principals’ 
reports. Only 48% of all schools (and 38% of urban schools) reach the cut-point for 
disciplinarian consistency. Only 31% of schools are adequately predictable in their responses to 
behavioral offenses. 

The “adequacy composite” percentages in Table 4.3 indicate the percentage of schools that 
were above our “adequacy cut-point” for 70% or more of the five indicators examined. The 
bottom line is this: Only 10% of our nation’s schools report using what we consider to be 
minimally adequate discipline practices. The remainder fail to employ available and acceptable 
methods to promote desired behavior or to diminish misconduct, or they fail to apply consistent 
and predictable disciplinary responses. The potential to improve practice in these respects may 
be great. 

Summary: Discipline policies and practices. The typical school has rules about dangerous 
behaviors, communicates those rules, and may apply severe consequences when these rules are 
broken. Of all schools, 94% have written rules or policies about weapons, 96% provide written 
copies of their rules to students and parents, and 97% of schools suspend or expel a student for 
possessing a gun. In view of the nearly universal existence of rules against weapons, it is 
unlikely that further school violence involving weapons can be prevented or reduced simply by 
introducing additional rules. Suspension or expulsion are used by fewer schools as a 
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Table 4.3 

Percentage of Schools with School-Wide Disciplinary Practices Judged Adequate According to Several Criteria, by School Level 
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consequence for other, more frequent, undesirable student behaviors such as tobacco possession, 
fighting, the use of profane or abusive language, and truancy - but the percentages of schools that 
suspend or expel for these offenses is still high. Suspension or expulsion is used for a wide range 
of offenses, apparently often without affording the student a hearing. 

Larger percentages of schools rely on punitive responses to misbehavior than on positive 
reinforcement of desirable behavior, and this imbalance is larger in high schools. For example, 
although more than 80% of high schools report using after-school detention, withdrawal of 
privileges, suspension, and the like; fewer than 70% use activity reinforcers, job or privilege 
reinforcers, and material reinforcers for desirable behavior. Because such reinforcers can be 
expected to work not only for younger students but also for older students, schools with students 
in higher grade levels may often be overlooking sources of regulation of student behavior. 

A minority of schools use what we consider to be minimally adequate discipline practices. 
The majority fail to employ available and acceptable methods to promote desired behavior or to 
diminish misconduct, or they fail to apply consistent and predictable disciplinary responses. The 
potential for making school disciplinary practices more responsive and consistent appears great. 

Quality of Discretionary Activities or Programs 

Table 4.4 shows the means on each intensity and fidelity measure, by program type. Across 
all program types, the average level of intensity and fidelity to good practice of school-based 
prevention activity is characterized by the descriptions in the following list: 

• One or more persons is conducting it from time to time-, 

• It employs 71% of the content elements identified as representing best practices; 

• It employs 54% of the methods elements identified as representing best practices; 

• It involves 32 sessions or lessons (although there is a large range across activities of 
different types); 

• It lasts about 25 weeks; 

• Both students and staff participate about once per week; 

• 41% of the school's students participate or are exposed; 

• There are approximately 4 program providers per 100 students in the school; and 

• If it is a classroom or a school-wide activity, it operates nearly all year. 

Although direct comparison across program categories is complicated by measures that are 
not strictly comparable, where comparisons are possible they imply differences in intensity or 
fidelity to good practice across categories. Classroom-level programs (categories 6 and 7) enjoy 
the highest level of use, e.g., they are more likely to be used by one or more persons on a regular 
basis. Mentoring, tutoring, or coaching as well as school planning activities also enjoy relatively 
high levels of use. The levels of use of security and surveillance and family programs are lowest. 
Prevention curricula stand out as employing particularly high proportions of identified best 
practices for content (81% on average), but prevention curricula on average employ only half 
(48%) of the identified best practices for instructional method. The counseling methods (other 
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than behavioral or cognitive-behavioral) used in schools to prevent or reduce problem behavior 
stand out as particularly poor in terms of their use of best practices for methods (only 33% of 
identified best practices used). This may explain why evaluations of counseling programs have 
not generally shown generic counseling to be effective (D. Gottfredson et al., in press). 

Table 4.4 shows that the mean number of sessions differs greatly for different categories of 
prevention activity, with family programs involving an average of 7 and improvements to 
classroom instructional methods an average of 101 sessions. Mentoring/tutoring activities 
involve a relatively large average number of sessions (47), and prevention curricula involve 28 
lessons on average. In terms of duration, school-wide planning and security activities tend to last 
longer (generally more than a year) than do services aimed at individual students. Of shortest 
average duration are recreational and other enrichment activities and services to families. The 
mean frequency of student participation ranges from about twice per month for family programs 
to more than once per week for behavioral programming. As might be expected, more students 
are exposed when the program is a school-wide climate change program (categories 8 and 9), and 
many fewer students are exposed on average in family programs and other individually-targeted 
programs such as mentoring and tutoring. School-wide programs to improve intergroup relations 
and encourage school-community linkages on average involve by far the largest number of 
providers (relative to the number of students in the school). Counseling programs involve the 
lowest ratio. 

Mean levels of intensity, exposure and use of best practices generally do not differ much by 
school level (not tabled). The only exception is that middle/junior high programs involve fewer 
sessions or lessons on average than do the high school programs (32, 25, and 37 for elementary, 
middle/junior high, and senior high schools). Evidence presented in Chapter 3 showed that 
middle/junior high schools operate a larger number of different programs than do elementary and 
high schools, but the available evidence does not imply correspondingly greater average intensity 
at the middle school level. 

More differences in the quality of programming exist across school locations (see Appendix 
Table H4. 1 8). Prevention activities in urban schools make use of a higher proportion of best 
practices (methods) than other schools. Activities in rural schools involve a lower level of 
involvement of school personnel (“level of use”) than do other schools. Also, activities in rural 
schools involve a lower level of student participation and operate less frequently than do 
activities in urban schools. 

Ratings of the Adequacy of Intensity and Fidelity to Good Practice 

The information about program quality provided earlier in this chapter provides a useful 
description of facets of prevention activity quality. It provides “normative” information in much 
the same way that tables of average body weights of men and women provides information about 
those populations. But we desired a way to go beyond that form of basic description to report on 
the “adequacy” of prevention programming. Just as tables of so-called normal or desirable 
weights provide guidelines against whether a person may be judged over weight, we sought a 



guideline or set of benchmarks by which prevention activities could be judged. This is akin also 
to what is sought in educational measurement when minimum competency standards are devised 
against which a student’s achievement can be compared. Judgment is required to develop such 
benchmarks. 

To devise “adequacy” guidelines for the present purpose, we decided that a useful criterion 
would be whether or not an activity could reasonably be expected to achieve a reduction in 
problem behavior or an increase in safety if it failed to meet a guideline. We made judgments 
about each available facet of program quality separately for each category of prevention activity. 
The two principal authors independently indicated (based on their understandings and 
interpretations of available research and information about practice in each area) the level that 
each indicator would have to reach in order to be expected to produce a measurable effect. 
Discrepancies between the judgments of the two raters were discussed and resolved. Tables 4.5 
and 4.6 show the minimum criteria necessary to be judged “adequate” on each dimension of 
program intensity and adherence to best practices. Table 4.5 shows thresholds for level of use 
and best practices with respect to content and method that were applied to all categories of 
activity, and Table 4.6 shows the separate thresholds for other facets of program quality that were 
applied to different categories of activity. 

Table 4.5 

Common Criteri£ Used_ to Judge Adequacy of All Categories of Prevention Activities^ 

Dimension Criterion 

Level of use by school personnel One or more persons is conducting activity on a regular 

basis 

Best practices: content Uses 70% or more of identified best practices 

Best practices: methods Uses 70% or more of identified best practices 



Table 4.6 

Criteria Used to Judge Adequacy of Prevention Activities That Differ According to Activity 



Category 

Dimension and Category Criterion 

Number of lessons/sessions 

Prevention curriculum, instruction, or training 2:16 

Mentoring 2 : 52 

Tutoring; Recreation, enrichment, leisure 2 : 26 

Improvements to instructional practices/methods 2 : 30 

External personnel resources for classroom 2 : 25 

continued . . . 
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Table 4.6 (continued) 

Criteria Used to Judge Adequacy of Prevention Activities That Differ According to Activity 
Category 



Dimension and Category 


Criterion 


Duration 

Prevention curriculum, instruction, or training; Counseling, social 
work, psychological, or therapeutic activity; Tutoring; 

Recreation, enrichment, leisure 


Longer than a month 


Mentoring 

Planning structure or management of change; Security and 
surveillance 

Frequency of participation - students 


At least one school year 

More than one full 
school year 


Culture, climate or expectations; Intergroup relations and 
school-community interaction; Planning structure or management 
of change 


At least 2-3 times per 
month 


Prevention curriculum, instruction, or training; Counseling, social 
work, psychological, or therapeutic activity; Mentoring, tutoring, 
coaching, apprenticeship; Recreation, enrichment, leisure; 
Services/programs for family members; External personnel 
resources for classroom 


At least weekly 


Improvements to instructional practices or methods , 

Behavioral programming or behavioral modeling; Security & 
surveillance 


More than once per week 
At least daily 


Frequency of participation - staff 

Culture, climate or expectations; Intergroup relations and 
school-community interaction; Planning structure or management 
of change 


At least 2-3 times per 
month 


Security and surveillance 
Frequency of operation 


At least daily 


Culture, climate or expectations; Intergroup relations and 
school-community interaction; Planning structure or management 
of change; Security and surveillance 

Proportion students exposed or participating 


Continually throughout 
the year 


Culture, climate or expectations; Intergroup relations and 
school-community interaction 


* 70% 


Youth participation in discipline 


> 1 0% or referrals 
handled by student court 
or through peer 
mediation 
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We then compared each activity against each of the adequacy criteria. For each of the 14 
categories of prevention activity, Table 4.7 shows the proportion of activities judged adequate 
according to each criterion of adequacy. For example, the first entry under column 1 (prevention 
curriculum, instruction, or training) means that 52% of activities in this category exceeded the 
adequacy threshold for level of use (at least one person is conducting the activity on a regular 
basis). The second entry in this column means that 76% of prevention curriculum, instruction, or 
training activities employed at least 70% of the identified best practices for content; the third 
entry means that 27% of these activities employed at least 70% of the best practices for method; 
and so on. The fourth entry in column 1 may be interpreted as meaning that 50% of prevention 
curriculum, instruction, and training activities offered enough lessons that it could to be expected 
to produce a measurable difference in a problem behavior outcomes (and that 50% did not have 
enough lessons). The last entry in each column shows the average proportion of quality 
dimensions that exceeded the adequacy criteria. The entry of .57 for column 1 shows that the 
mean proportion of the six adequacy criteria met by prevention curriculum, instruction, or 
training activities was .57. 

The dashes in Table 4.7 indicate facets of program quality for which it was not possible to 
establish adequacy criteria - either because there was no basis in research to specify a criterion, 
or because the quality dimension was not measured. 

The overall quality of prevention programs in schools is low. For all types of programs, the 
mean proportion of adequacy criteria met is only .57. This means that for the average activity, 
only 57% of the indicators of quality or quantity were judged to be sufficiently strong to be 
expected to lead to a measurable difference in the desired outcomes. The summary index ranges 
from a low of .42 for services or programs to family members to .73 for security or surveillance 
activities. 

Across all types of programs, the proportion of activities judged adequate ranged from a low 
of .33 for the use of best practices (methods) to a high of .75 for frequency of operation. The use 
of best practices (methods) had a low overall proportion adequate because several kinds of 
activities aimed directly at altering student behavior (counseling, mentoring or tutoring, 
behavioral programming or modification, and instruction) make little use of the identified best 
practices for methods. The proportion of activities meeting the adequacy criterion for the 
number of lessons or sessions was also low at .37. Activities involving the use of external 
personnel for classroom management or instruction rarely meet this criterion, and individual 
attention (mentoring or tutoring) and recreational programs also generally fall short on this 
criterion. On the other hand, high proportions of activities directed at security and surveillance 
or classroom organization and management operate continually throughout the school year, 
which was the criterion for adequacy on the “how often” dimension. 

In general, classroom- and school-level activities seem to be implemented with somewhat 
higher quality than activities targeting individual students. Security and surveillance activities 
are the best-implemented (the mean of the six facets of adequacy is .73), partly because 95% of 
these activities operate continually throughout the year. School planning activities (average facet 
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1 able 4.7 

Proportion of Programs or Activities Judged Adequate on Each Dimension, by Program Type 
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Proportion of dimensions judged adequate 



adequacy of .71), classroom organization and management activities (average of .71 of criteria 
adequate), activities that involve youths in regulating student behavior (e.g., peer mediation, 
student courts; average of .69 of criteria adequate), activities that change the school climate 
(average of .64 of criteria adequate), and improvement to classroom instructional methods 
(average of .59 of criteria adequate) are all implemented with above-average quality. 
Individualized services - family services, behavior management, and counseling - were the most 
poorly implemented activities, with averages of .42, .47, and .45 of criteria adequate. 

Certain ratings of adequacy of implementation vary by school level. The adequacy of the 
frequency of student participation and the proportion of students participating decrease as the 
school level increases, and in both instances elementary schools differ significantly from high 
schools. This accords with experience in working with schools at different levels - as students 
become more autonomous they opt out of many school activities. 

Consistent with the analysis of the mean levels each quality indicator, the adequacy ratings 
also tend to be higher in urban and lower in rural areas. Programs in urban schools are judged 
adequate more often than other schools on the use of best practices (methods). Programs in rural 
schools are judged adequate less often than other schools on level of use by school personnel and 
the overall rating of adequacy. 

One interpretation of the summary index “proportion of dimensions judged adequate” is that 
it provides an optimistic assessment of the likely effectiveness of a program or activity. To see 
why this is so, consider a hypothetical instructional program. Suppose the program utilizes all of 
the identified best practices for method and content, and exceeds the threshold for number of 
lessons and duration, but no one implements the program on a regular basis and students almost 
never participate. This program would have a score of 4/6 or 67% of adequacy criteria met. But 
since the program is evidently designed well but essentially unimplemented, it cannot be 
expected to produce anything in the way of results. A failure to meet standards for adequacy for 
even one of the dimensions can potentially render an activity impotent. 

Variability in Program Quality 

Results presented above indicate that the quality of program implementation is variable and 
often poor. But this summary does not convey information about the large amount of variability 
in program implementation from activity to activity, even among activities of the same type. The 
discovery of great variability in program or activity quality is an important finding of the 
National Study of Delinquency Prevention in Schools. It implies that any type of prevention 
strategy can be well implemented, and that any type can be poorly implemented. 

Consider level of use, one of the indicators of program intensity that is measured in a 
parallel way for all categories of prevention activity studied. The percentage of variance in level 
of use that lies between program categories is only 5%. This means that most of the variability 
in this indicator is within program category. Even in indicators which are to a certain degree 
dependent upon program category for their measurement, most of the variability in the measure is 
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within category. The proportion of dimensions judged adequate, for example, has only 28% of 
its variance between category. 

A nontechnical and perhaps more intuitive way to convey this point is to show examples of 
specific programs of the same type which differ in their quality of implementation. Tables 4.8 
through 4. 10 show examples of high and low quality school planning, behavior management, 
and D.A.R.E. programs, based on descriptions from the Activity Coordinator Questionnaires. 

These tables illustrate how activities within each category vary considerably with respect to 
intensity and adherence to good practices. This is true even for highly standardized programs 
such as D.A.R.E. 



Table 4.8 



Low and High Quality^ School P^ ann ^ n S Interventions 



Intensity/Fidelity Measure 


Program A: School 
Planning Teams 


Program B: School 
Improvement Teams 


Level of use by school personnel 


One or more persons is 
participating in it from 
time to time 


One or more persons is 
conducting activity on a 
regular basis 


Duration 


One week 


At least a full school year 


Frequency of participation - 
students and staff 


one or twice per school 
year 


Daily 


Comments 


• The principal and a 
counselor are 
responsible for 
conducting the activity. 


• A broad spectrum of 
school staff, police, and 
community members 
are responsible for 




• Participants received a 
short demonstration in 
how to conduct the 
activity. 

• Participation is 
voluntary, and 
participants are not held 
accountable for 
conducting this activity. 

• The activity is not 
funded. 


conducting the activity. 

• Participants received 2- 
3 days training. 

• It is a required program, 
and participants are 
held accountable for 
conducting this activity. 

• The activity is funded 
through its school 
system budget and 
other external funds. 



Table 4.9 



Low and_ High Quality^ Behavior^ Modification Interventions 



Intensity /Fidelity Measure 


Program A: Alternative 
Classroom Education 


Program B: Behavior 
Modification Program 


Level of use by school personnel 


One or more persons has 
been trained 


One or more persons is 
conducting activity on a 
regular basis 


Best practices: content 


43% 


100% 


Best practices: methods 


0% 


88% 


Duration 


One month 


More than a full school 
year 


Frequency of participation - 
students 


Monthly or less 


More than once per day 


Proportion students exposed or 
participating 


8% 


3% 



) 

Table 4.10 

Low and_ High Quality D.A.R.E. Programs 



Intensity/Fidelity Measure 


Program A: D.A.R.E. 
Instructional Program 


Program B: D.A.R.E. 
Instructional Program 


Level of use by school personnel 


One or more persons is 
conducting activity on a 
regular basis 


One or more persons is 
conducting activity on a 
regular basis 


Best practices: content 


91% 


100% 


Best practices: methods 


0% 


100% 


Number of lessons/sessions 


5 


16 


Duration 


About a week 


Less than a half school 
year 


Frequency of participation - 
students 


Less than once a month 

\ 


Weekly 


Proportion students exposed or 
participating 


30% 


23% 
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Conclusion 



In Chapter 2 we showed that schools conduct many different activities aimed at reducing 
problem behavior and increasing school orderliness. This section more closely examined the 
quality of those activities. Using reports from almost 3,700 prevention activities in our nation’s 
schools, we examined the intensity of the activities and their adherence to good practice, as 
implied by accumulated knowledge from education, prevention, and evaluation research and 
experience. 

The quality of prevention activities in the nation’s schools is generally poor: The average 
prevention activity receives a passing grade on only 57% of the quality criteria examined. In 
general, individual prevention activities are not being implemented with sufficient strength and 
fidelity to be expected to produce a measurable difference in the desired outcomes. On the other 
hand, there is so much prevention activity underway at all levels that it is possible that multiple 
activities - each with small effects - may cumulate to make a substantial difference. However 
that may be, the poor quality of most prevention activity underscores the importance of 
establishing conditions in schools that are conducive to high quality implementation. Perhaps, 
for example, modifying programs to make them more “user friendly” or “goof proof’ would 
help. Perhaps more and better training might be required. Perhaps greater organizational 
support, such as feedback and coaching, solid principal support, or more organizational 
commitment might be necessary. More certain and greater amounts of funding might be 
required. The next section of the report explores these and other potential predictors of the 
quality of prevention activity implementation. 



Elsewhere (D. Gottffedson, in press; G. Gottffedson, Jones, & Gore, 1999) we have argued 
that some urban schools pose more challenge to prevention programming because they are more 
likely to lack the requisite organizational i nfr astructure to plan for and carry out high quality 
programs. And some schools serving areas of concentrated poverty and social disorganization 
have special difficulties because of the elevated needs of their student populations - which may 
require that more resources be directed to urgent needs that arise in an unpredictable manner. 

We had expected that we would find lower quality of implementation of prevention programs in 
urban schools. The data fail to confirm an expectation that urban location means poorer 
implementation than other locations. Instead, the adequacy ratings (as well as the number of 
programs attempted) are higher in urban and lower in rural areas. One possibility is that schools 
in the most disorganized urban settings in the sample did not participate in the surveys. The 
response rates were lower among urban schools than in other schools. Unraveling the influence 
of study nonparticipation and community characteristics will require better measures of 
community social organization and urbanicity than are currently available, but the next section 
will explore whether other features of programs or schools can explain the differences among 
schools in levels and quality of implementation. 



Although most of the variability in implementation quality lies within activity category, 
indicators of program quality do vary by type of prevention activity. In general, activities that 
aim to alter the school or classroom environment are better implemented than those aimed at 
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altering student behaviors or attitudes. Services or programs operated by schools for family 
members of students are generally weak (the average adequacy score across the three quality 
dimensions assessed was only .42). Security and surveillance activities are best implemented. 

These differences by program type do not imply that schools should abandon those types of 
activities that appear more challenging to implement. We reiterate that quality of program 
implementation varies far more within than between program categories. We find in the data 
examples of high quality and low quality programs of every type. Despite earlier conclusions (D. 
Gottfredson, 1997; D. Gottfredson et al., in press, 2000) about the kinds of preventive 
interventions that do and do not work, a well-implemented program of the type that has generally 
been found to be inefficacious may prove more effective than a poor implementation of a 
program type that has been found efficacious in earlier research. 

Earlier research has demonstrated that preventive interventions are less likely to produce 
desirable outcomes when they are implemented poorly. Research by Botvin and his colleagues 
summarized earlier showed that when less than 60% of Botvin’s Life Skills Training (LST) 
curriculum is delivered, the program has no measurable effect. It appears likely that the typical 
quality of prevention activity carried out in schools falls short even of the minimum level Botvin 
identified as necessary. LST is currently the subject of efforts at replication with training and 
technical assistance being provided to 142 schools in 35 sites as part of the Blueprints project led 
by Delbert Elliott at the University of Colorado with support from the U.S. Office of Juvenile 
Justice and Delinquency Prevention and the assistance of Gilbert Botvin. A number of 
difficulties in achieving the intended levels of implementation have been encountered (Center for 
the Study and Prevention of Violence, 2000), including instruction by physical education 
teachers who are unfamiliar with teaching a curriculum, limited instructor classroom 
management skills, large classes, distracting settings or settings that are usurped for other 
activities, teachers who are not prepared for or committed to taking on a new instructional role, 
teacher turnover and the loss of trained instructors due to illness or job change, deviations from 
the curriculum, supplementation of or replacement of material with other material, and failure to 
use the technical assistance (TA) which is available. If all of these difficulties are encountered in 
sites that have competed for the opportunity to receive TA and training, and been screened and 
selected on the basis of applications and feasibility visits to receive that training, imagine the 
difficulties that may occur in a school in which someone decides to teach a social skills module 
using whatever curriculum was available and without the TA and training. 

A summary of the results on the quality of prevention programs in the nation’s schools is 
provided in the form of a “report card” in Table 4.11. Each prevention activity can be 
characterized by the percentage of the quality dimensions examined that were rated “adequate.” 
These percentages are mapped into letter grades using the traditional 90% and above = A, 80% - 
89% - B, and so on. Overall, 47% of activities receive a failing mark according to this report 
card; 1 8% earn an A. We hesitate to offer this simple report card summary, because of the 
considerable amount of both complexity and judgment that entered into the calculation of grades, 
and because we assume that this report card summary may be all that is communicated about the 
present inquiry in secondary accounts about it. At the same time, none of the decision rules upon 
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which the summary is based is capricious and we believe where there is error it lies on the side of 
leniency. These grades are lenient because in principle it is possible for a program to fail in the 
real world (i.e., to be ineffective) if it fails to meet even one quality criterion. Therefore, we 
assume that some fraction of programs that would earn an A, B, or C by the calculus used to 
assign the Table 4.1 1 grades are weaker than the letter grades suggest. In the final analysis, the 
grades in Table 4.1 1 emphasize that there is much room for improvement in the quality of activity 
to prevent problem behavior in schools. 



Table 4. 11 



Percentage Distribution of Overall Activity Grades, by Location 



Grade 


Percentage of quality 
dimensions rated 
“adequate” 




Location 




Urban 


Suburban 


Rural 


All 

locations 


A 


90% - 100% 


20 


18 


15 


18 


B 


80% - 89% 


12 


10 


11 


11 


C 


70% - 79% 


13 


11 


10 


11 


D 


60% -69% 


15 


13 


12 


13 


F 


< 60% 


40 


48 


52 


47 


Total 




100 


100 


100 


100 



Note. Grade maps into the percentage range of quality dimensions judged to be adequate. 
Percentages awarded each grade add to 100% down the columns, within rounding error. 



More sophisticated research is required to inform us about the relative contributions of 
program content and method on the one hand and quality of implementation on the other in 
determining effectiveness. In the interim, however, it seems wise to recommend that schools 
should concentrate their efforts on improving the quality of what they are already doing. This 
may result in more improvement in program outcomes than adopting new program models or 
switching to different models of preventive intervention. At the same time, improving 
implementation across the board may require that we develop processes or mechanisms to boost 
the quality of prevention program implementation. 

We turn in the next chapter to an examination of the characteristics of programs, 
populations, providers, and organizations related to the quality of implementation. 
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Predictors of Quality of Program Implementation 



This chapter examines correlates of the quality of prevention activity implementation. The 
indicators of quality described in the previous chapter are the criterion measures. The chapter 
begins with a summary of hypotheses about predictors of sound program implementation. The 
remainder of the chapter summarizes tests of these hypotheses as well as additional explorations 
that were not driven by specific hypotheses. We examine, for example, whether “packaged” or 
“multi-component” programs are implemented with greater or lesser strength than “home-grown” 
or unitary activities, and we will examine the role of school-based planning in the 
implementation process. In the present chapter we examine data about quality at the level of 
individual programs or activities. Then, in the following chapter, we examine data about the 
quality of prevention activities at the school level and examine information about the relations 
between school characteristics and program quality. In performing these tests and explorations, 
each of the data sources (Principal, Teacher, Student, Program Coordinator Questionnaires, and 
archival data) are used to identify characteristics of programs, populations, providers, and 
organizations related to the quality of program implementation. 

Factors Hypothesized to Leading to Successful Program Implementation 

The following categories of factors are hypothesized on the basis of prior research and 
experience to be linked to the successful implementation of prevention programs. 

1 . Organizational capacity (morale, staff stability, history of failed or successful programs in the 
past). 

Better morale, more stable staff, and a history of successful program implementation in the 
past is expected to go with better current implementation. In contrast, low morale, high staff 
or principal turnover, and a history of failed programs is expected to go with poor 
implementation. 

2. Leadership and staff traits and past accomplishments. 

Implementation is expected to be better in schools in which principals report that they display 
behaviors associated with effective leadership and where they are perceived by others as 
effective leaders. Schools where principals or program implementers have a record of 
accomplishment in the past are expected to be more successful in what they currently 
implement. And programs implemented by more conscientious implementers in schools led 
by more conscientious principals are expected to be better implemented. 
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3. Budget and resources. 



Lack of adequate budget or resources is expected to thwart successful program 
implementation, and adequate budget and resources is expected to promote quality 
implementation. 

4. Organizational support (training, supervision, principal support). Extensive and high quality 
training is expected to promote high quality and extensive implementation, whereas lack of 
training and poor training is expected to lead to weak or poor quality implementation. Direct 
and more extensive supervision is expected to lead to higher quality and more complete 
implementation, whereas lack of supervision is expected to allow low quality and limited 
implementation. Principal support for an activity is expected to lead to more extensive 
implementation and to higher quality implementation. 

5. Program structure - manuals, implementation standards, quality control mechanisms. 

Greater structure is expected to lead to higher quality implementation and implementation 
that more closely follows a plan for what should be implemented. Implementation manuals 
can provide scaffolding for implementers by providing structure, an organization, and a plan 
for what to do as well as guidance on how to do it. Prepared materials, such as handouts, 
overhead masters, and videotapes, can make implementation easier and deviation from 
intended content less likely. Statements of standards for implementation provide the persons 
implementing a program with.a basis for. determining whether what is being done is good 
enough. And quality control mechanisms such as procedures for monitoring progress, review 
of progress, and worker supervision are expected to promote better implementation by 
focusing attention on how well implementation is being done. 

6. Integration into normal school operations, local initiation, and local planning. 

The extent to which program design choices are integrated with normal school operations is 
expected to have consequences for implementation. Better integration of activities with the 
regularities of the school is expected to lead to more enthusiastic and widespread adoption of 
prevention practices within a school. Schools do certain kinds of things as a matter of 
routine. Preeminently, schools conduct instruction organized in classrooms. Schools employ 
teachers, supervised by principals, to carry out instruction. Although schools often employ 
other categories of workers, including counselors, nurses, clerical personnel, and 
maintenance workers, teachers constitute most of the school workforce and they are the 
personnel who most often and most directly interact with students. Schools also sometimes 
utilize the services of volunteers or other persons not in the employ of the school. Unlike 
regular school employees, the timing, duration, and extent of involvement of these external 
personnel is only weakly controlled by the school. The extent to which a prevention activity 
is carried out by regular school employees in the conduct of their accustomed work (i.e., 
teaching), the more widely implemented it is likely to be. Other things being equal, 
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administrators are more likely to implement extensively and well activities that involve 
administration, teachers are more likely to implement activity that involves teaching in their 
classrooms, counselors are more likely to implement activity that involves counseling in their 
offices, and muses are more likely to implement activity that resembles traditional nursing 
activities. 

When activities or arrangements are selected, devised, or planned by persons within a school 
organization, they are expected to be more acceptable to persons within the school. When 
locally planned or initiated, activities are (by definition) not imposed upon a school and 
impulses to resist adoption or implementation which are sometimes triggered by programs 
imposed upon a school are less likely to be evoked. 

When school personnel use information about what and how to implement activities derived 
from researchers, experts, publications, and other sources, they are expected to incorporate 
more best practices and to emulate successful models more fully because they are more likely 
to have the information needed to do so. 

7. Program feasibility (match between program design features and regular activities of schools, 
few obstacles). 

Some activities or arrangements are expected to encounter obstacles to implementation. 
Activities that occur after the end (or before the beginning) of the regular school day or on 
weekends will be more difficult to implement because they are outside of regular work hours, 
activities that require transporting students away from the school will be more difficult to 
implement routinely than those that take place in the school, and activities that are difficult to 
carry out with a classroom-sized group of students in a 30 to 50 minute period are unlikely to 
be sustained. 

8. Level of disorder. 

It is expected that high levels of disorder in a school will make everything more difficult to 
implement. High levels of disorder may provide impetus to the adoption of prevention 
programs, activities, and arrangements. But other things being equal, the distractions and 
emergencies of a disorderly environment are expected to undermine the quality of 
implementation of such programs, activities, and arrangements. 

Measures of Hypothetical Predictors of Program Quality 

Measures of the hypothetical predictors of program quality are derived from reports by 
principals, teachers, implementers, students, and archival sources. Table 5.1 shows the names 
and sources of each of the different indicators of the predictors. It also shows the number of 
items and, as appropriate, an alpha individual-level reliability and an estimated lambda reliability 
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Table 5.1 

Measures of Hypothetical Predictors of Program Quality 



Category and predictor scale or item name 


Source 


N 

items 


a a 


X. 


Organizational capacity 


Morale 


TQ 


11 


.81 


.88 


Organizational focus 


TQ 


16 


.94 


.86 


School amenability to program implementation 


PQ2 


9 


.76 


- 


School amenability to program implementation 


AQ 


11 


.81 


.69 


Faculty-administration obstacles to implementation 


PQ1 


12 


.76 


- 


School capacity for program development 


PQ1 


6 


.55 


- 


Open identification of problems 


PQ1 


3 


.55 


- 


Teacher-principal communication 


PQ1 


2 


.59 


- 


Teacher turnover 


PQ1 


l b 


- 


— 


Program or activity staff turnover 


AQ 


1 


- 


.43 


School size 


PQ1 


1 


- 


- 


Leadership and staff competencies, traits, past accomplishments 


Administrator leadership 


TQ 


12 


.84 


.88 


Leadership behavior 


PQ2 


19 


.90 


- 


Accomplishment record of principal 


PQ2 


7 


.70 


- 


Accomplishment record of activity coordinator 


AQ 


12 


.84 


— 


Conscientiousness of principal 


PQ2 


20 


.90 


— 


Conscientiousness of activity coordinator 


AQ 


20 


.91 


— 


Non-delegation of responsibility by principal 


PQ1AD 


l c 


- 


- 


Broad principal span of control 


PQ1AD 


l d 


— 


— 


Budget 


Funding for program assured next year 


AQ 


1 


- 


.40 


Budget control over project activities 


AQ 


1 


- 


.44 


Organizational support 


Amount of training in classroom management/instructional 


TQ 


1 


— 


.63 


methods 


Amount of training in preventing student problem 


TQ 


1 





.70 


behaviors 


Quality and quantity of training in discipline 


PQ2 


8 


.91 




Amount of training in activity/program 


AQ 


3 


.67 


.52 
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Table 5.1 (continued) 

Measures of Hypothetical Predictors of Program Quality 



Category and predictor scale or item name 


Source 


N 

items 


ct a 


1 


Quality of training in activity /program 


AQ 


6 


.87 


e 


Monitoring of conformity of discipline practices with 
policy 


PQ2 


1 


— 


— 


Principal’s performance appraisal depends on discipline 
management 


PQ2 


1 


— 


— 


Supervision or monitoring of implementation of program or 
activity 


AQ 


3 


.55 


.49 


Principal support for program or activity 
Program structure 


AQ 


1 


' 


.44 


Standardization 

Integration into normal school operations 


AQ 


5 


.72 


.45 


Planning 


TQ 


9 


.62 


.84 


Local responsibility (school insiders) for program initiation 


AQ 


14 


.82 


.50 


School district responsibility for program initiation 


AQ 


' 4 


.77 


.57 


Variety of information sources used in selection of 
discipline practices 


PQ2 


7 


.68 


— 


Variety of information sources used to select program or 
activity 


AQ 


7 


.70 


.51 


Amount of provider’s job related to program or 
activity 


AQ 


1 


— 


.24 


Activity is part of regular school program 


AQ 


1 




.27 


Provider is full-time 


AQ 


1 


- 


.40 


Paid workers deliver program or activity 


AQ 


1 


- 


.44 


Local initiative versus Safe and Drug Free Schools and 
Communities coordinator initiative 


PQ2 


1 


— 


— 


Local development of discipline practices 
Program or activity feasibility 


PQ2 


5 


.68 


— 


Obstacles to program implementation 


AQ 


12 


.74 


.44 


Activity occurs during the school day f 


AQ 


1 


- 


.52 


Activity occurs in the early evening (6:00 - 9:00 p.m.) f 
Level of disorder/problem behavior 


AQ 


1 




.59 


School safety, teacher perspective 


TQ 


8 


.94 


.75 


School safety, student perspective 


SQ 


13 


.80 


.86 



continued. . . 
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Table 5.1 (continued) 

Measures of Hypothetical Predictors of Program Quality 



Category and predictor scale or item name 


Source 


N 

items 


cc a 


L 


Classroom orderliness 


TQ 


14 


.92 


.79 


Teacher victimization 


TQ 


8 


.61 


.72 


Student victimization 


SQ 


7 


.61 


.68 


Selectivity 


PQ1 


5 


.86 


- 


Problem student magnet 


PQ1 


3 


.81 


- 


School crime 


PQ2 


5 


.68 


- • 


Gang problems 8 


PQ2 


2 


.38 


- 


Last-year variety drug use 


SQ 


16 


.87 


.88 


Delinquent behavior 


SQ 


13 


.84 


.78 



Note, a = alpha reliability for individual-level measure. A. = estimated reliability of school-level 
aggregate; calculated from unweighted data excluding schools with fewer than 10 students (or teachers) 
unless 70% of sampled students (teachers) responded. PQ1 = phase 1 principal questionnaire, PQ2 = 
phase 2 principal questionnaire, AQ = activity coordinator questionnaire, TQ — teacher questionnaire, SQ 
= student questionnaire, PQ1AD = phase 1 principal questionnaire activity detail booklet. 
a Value shown for PQ2 is the median alpha for elementary and secondary schools. 
b Ratio of new teachers this year relative to the total number of teachers. Although the calculation of this 
item is based on responses to two questions, there is only a single indicator of turnover in the principals’ 
reports. 

c Percentage of named prevention activities for which the principal listed him/herself as the only person 
who can provide information. 

d Percentage of named prevention activities for which the principal listed him/herself as one of the 
individuals who can provide information. 

' Questions about quality of training were not answered by respondents who indicated that there was 
none. Too few schools had multiple responses on training quality to calculate dependable reliability 
estimate for the school level. 

f Respondents indicated when the activity occurred using a list of possibilities, including weekends and 
immediately after school. Only the two time intervals listed here were empirically related to program 
quality. 

8 Alphas differed greatly for elementary and secondary schools (elementary school principals tended to 
report few gang problems). Elementary a = .23, secondary alpha = .54. 
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for the average school.' Alpha reliability cannot be calculated and therefore is not shown when 
construct is represented by only one item per individual respondent. Lambda reliability cannot 
be calculated and therefore is not shown when there is only one individual (or a very small 
number of individuals) describing each school. In such cases differences among schools are 
confounded with individual differences in views or opinions about a school and we cannot 
estimate the proportion of variance that lies between schools. For the present purposes, the 
school-level reliability of measures - the column headed X. - is of importance. Just as a is 
conceptually an average inter-item correlation stepped up according to the number of items in a 
scale, so also is lambda-hat conceptually the intraclass correlation (p) stepped up according to the 
number of respondents per school. It depends on the size of the intraclass correlation and the 
number of observations per school and so reflects the relative amount of variance between 
schools as well as the size of the sample. 

The information on alphas from Table 5.1 suggests the following interpretations with respect 
to the measurement of specific activities or programs and individuals: 2 (a) For some measures - 
perceived morale, perceived organizational focus, perceived amenability to program 
implementation, perceived administrator leadership, principals’ self-reported leadership 
behavior, accomplishment record of activity coordinators, conscientiousness (principal and 
activity coordinator), quality and quantity of training in discipline, quality of training in 
activities/programs, perceptions of local responsibility for program initiation, perceptions of 
school safety (teacher and student), classroom orderliness, principals’ reports of school 
selectivity and of the attraction of problem students, and student self-reports of drug use - the 
alphas are relatively high (above .7) implying that respondents tend to provide consistent 
accounts of their own behavior or how they see the school, (b) In contrast, for some measures 
alphas are considerably smaller. These measures are either event scales which would not be 
expected to have high internal consistency, or they have fewer than five items. And (c) for 
variables represented by single items no estimate of individual-level reliability is available. 

The information on lambda-hats from Table 5.1 suggests the following interpretations with 
respect to the measurement of school characteristics: (a) For some measures - Morale, 
Organizational Focus, Administrator Leadership, Planning, Safety (both teacher and student 
reports), Classroom Orderliness, Last-Year Variety Drug Use, and Delinquent Behavior - school 



'The item content or illustrative item content of the scales listed in the table may be found in 
Appendix E. Additional information about reliability, including intraclass correlations (p), is 
provided in Appendix F. Tables of correlations among measures are shown in Appendix G. 
Descriptive information, generally tables showing means for the measures by school level and 
location and (for activity questionnaires) by program category, is provided in Appendix H. 

2 Some measures are not intended to apply to individuals. For example, school safety should be 
considered to be a characteristic of a school rather than of individuals. When scored at the 
individual level, a score on a safety scale reflects individual differences in perception as well as 
the influence of the school environment on these perceptions. 
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characteristics appear to be reliably measured as lambda-hats are all .75 or above, (b) In contrast, 
for some measures - program or activity staff turnover, whether funding is assured for the 
following year, degree of budget control over activities, the degree of principal support for 
programs or activities, Standardization, amount of provider’s job related to the program or 
activity, whether activities are part of he regular school program, whether the provider is a full- 
time worker, whether paid workers deliver the program or activity, and Obstacles to Program 
Implementation - school characteristics are not measured with high reliability as lambda-hats are 
all .45 or below. Low school-level reliability is to be expected when there is a great deal of 
heterogeneity within schools. For example, if some programs or activities involve a great deal of 
local responsibility for program initiation and others within the same school involve very little 
such responsibility, then within-school variability can be high relative to between school 
variability. This appears to be the case. The individual-activity-level reliability (alpha) for Local 
Responsibility for Program Initiation was a relatively high .82, but the school-level reliability 
(lambda-hat) for this scale was a more modest .50. 



In general, the school-level assessments based on teacher or student surveys using scales 
from the Effective School Battery (G. Gottffedson, 1984/1999; Morale, Administrator 
Leadership, Planning, Safety, Victimization), which were developed to measure school 
characteristics, are satisfactory. This is also true of the Organizational Focus scale (G. 
Gottffedson & Holland, 1997) which was developed to measure differences among 
organizations, and also of the Last-Year Variety Drug Use, and Delinquent Behavior scales (G. 
Gottffedson & Gottffedson, 1999) which were developed to measure individual differences but 
are shown in Table 5.1 to produce reliable measures of schools as well 



Correlations Between Characteristics of Activities and Indicators of 
Activity Quality 



We turn now to a summary of the relations between the hypothesized predictors of program 
quality and our indicators of quality. Additional empirical links are also examined, but we warn 
the reader that the lack of explicit hypotheses makes this extended review something of a fishing 
expedition. The following paragraphs review correlations between characteristics of activities 
and indicators of activity quality. 3 The correlations described are based on unweighted sample 



3 Appendix Tables H5.1 through H5.3 show correlations between the hypothesized predictors of 
program quality and the indicators of program quality. The tables are organized according to the 
following general categories: Characteristics of the activity (Appendix Table H5.1), 
characteristics of the program coordinator (Appendix Table H5.2), and the origin of the activity 
and its funding sources (Appendix Table H5.3). These tables use data only ffom the activity 
questionnaires («=3,702). Recall that certain quality indicators were scored only for certain types 
of activities. Similarly, certain predictors are meaningful only for certain types of activities. The 
range of numbers of activity questionnaires on which each correlation is based are shown for 
each quality indicator in the table column heads. 
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data, and so should technically be taken as estimates of correlation in our sample rather than in 
the population of prevention activities. 4 The following chapter summarizes results from all 
sources, including correlations between average activity or program quality at the school level 
and other school-level variables. An examination of school-wide disciplinary practices is 
deferred until school-level variables are examined. Here we examine the empirical associations 
between characteristics of prevention activities and their quality of implementation. We begin 
with the results concerning hypothesized predictors, and then examine results for other variables. 

Evidence About Hypothesized Predictors 

Capacity. As hypothesized, program coordinators’ views of the schools’ amenability to 
program implementation was positively and significantly correlated with six of the eight 
indicators of program quality. Correlations ranged from .00 to .12, Mdn = .08. Although 
correlations are small, their direction supports the hypothesis. In contrast, the median correlation 
between program or activity staff turnover and the eight measures of activity quality is .04. 
Correlations range from -.02 to .07, and confidence intervals for only three of the positive 
correlations do not include zero. Contrary to the hypothesis, activities where staff have been 
replaced because they left or were dismissed are of slightly better quality than those with more 
stability. Possibly poor staff are replaced by better implementers. Correlations are very small, 
however (see Appendix Table H5. 1). 

Program coordinator accomplishments and traits. More conscientious program 
coordinators, and coordinators with a record of more program-related accomplishments 
coordinate programs with better implementation. For conscientiousness, the confidence intervals 
for correlations with four of the eight quality indicators are positive (.04 to .08) and do not 



4 Statements about statistical significance and confidence intervals are based on an assumption of 
simple random sampling. Once weights adjusting for sampling probabilities and nonresponse 
became available, we recalculated correlations and significance levels for the variables examined 
in Appendix Table H5.3 (i.e., correlations between activity quality and origins and sources of 
funding) to leam whether the application of weights and the use of resampling to estimate 
standard errors would have led to different interpretations. Appendix Tables H5.4 through H5.6 
show side-by-side comparisons of correlations estimated with and without weights. Standard 
errors estimated by resampling are usually slightly larger than those estimated using the 
assumption of simple random sampling, and the correlations occasionally bounce a bit when 
weights are applied. But conclusions would not generally differ depending on the estimation 
method used. For example, the largest correlation in Appendix Table H5.4 is the .24 correlation 
between the use of best practices (methods) and school insider responsibility for starting the 
program. The correlation rounds to .24 whether weighted or unweighted data are used; the 95% 
confidence interval for the correlation is .193 - .284 under the assumption of simple random 
sampling and is .163-. 309 when the standard error is estimated by resampling. Because results 
are so similar, in examining correlations, we decided not to apply weights or to use resampling to 
estimate sampling errors. 
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include zero. For accomplishment, the confidence intervals for correlations with five of the eight 
quality indicators are positive (.05 to . 1 3) and do not include zero. One correlation is negative 
(-.11) and significantly different from zero: For the subset of program types for which frequency 
of staff participation was measured, staff participate less in the programs run by coordinators 
with a record of more past accomplishments. Although all are small in size, these correlations 
generally suggest that selecting coordinators who are higher in conscientiousness and who have a 
track record of past accomplishments would improve the quality of program implementation. 

The associations are in the small range, however (see Appendix Table H5.2). 

Budget and support. We hypothesized that programs with more secure funding and 
programs in which the coordinator had more control over the budget for the activity would be 
better-implemented. These hypotheses are generally not supported (Appendix Table H5.3). 
Programs whose funding is more secure for the next school year are more likely to expose a 
greater proportion of students and have slightly higher ratios of providers to students, but they are 
also slightly less likely to make use of “best practices.” The median correlation of assured 
funding for the next year and the eight measures of quality is only .02. Programs whose 
coordinators have more control over the budget are more likely to expose a greater proportion of 
students, but this is the only association out of eight possible for which the confidence interval 
for the correlation (.12) does not include zero. The median correlation is only .02. 

Organizational support. The level of supervision, quality of training, amount of training, 
and principal support for the prevention activity were hypothesized to predict the quality of 
implementation. The evidence generally supports the importance of these four variables. Of. the 
32 relevant correlations, 25 are statistically significant and in the expected direction. No 
significant result is in the unexpected direction. Moreover, the correlations are often moderate in 
size. For level of supervision, correlations with the eight quality criteria range from .00 to .25, 
Mdn = .14; for training quality correlations range from -.03 to .15, Mdn = .10; for amount of 
training correlations range from .02 to .18, Mdn = .14; for principal support correlations range 
from -.01 to .21, Mdn = 13 (see Appendix Table H5.1). Table 5.2 shows mean scores for 
selected indicators of program quality as a function of those indicators of organizational support 
that best predicted quality. 

Standardization. Standardization of program materials and methods is also related to higher 
quality implementation, supporting the hypothesis. Programs scoring higher on the 
Standardization scale (i.e., activities with manuals; that include reproducible materials; use 
videos, films, etc.; provide lists of materials to be used; and specify the activities to be carried out 
are used more regularly) reach more students, and incorporate a greater percentage of “best 
practices” than less structured programs. Correlations for six of the eight measures of program 
quality are positive and their confidence intervals do not include zero (range of correlations = 

- .03 to .23, Mdn = .08). See Appendix Table H5. 1 for details. Table 5.3 shows mean scores for 
selected indicators of program quality as a function of those indicators of standardization that 
best predicted quality. 
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Table 5.3 . v 

Activity Quality by Standardization and Time 

. Proportion "best practices" used — 

content Intensity Frequency of participation — staff 

Activity characteristics M C\ N M Cl N M Cl N 
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Integration into normal school operations. The hypotheses about integration into normal 
school operations received stronger support from the data. When school insiders have greater 
responsibility for initiating a program, the program is more often implemented in a higher quality 
fashion. Correlations between school insider responsibility and seven of the eight quality 
indicators had non-zero correlations (ranging from .09 to .24). The remaining correlation 
between this measure of integration into school operations was in the opposite direction to that 
hypothesized, - .07. The median correlation between insider responsibility and measures of 
quality was .15. Programs for which the school district or researchers had more responsibility for 
initiation also have generally positive, non-zero correlations with the quality indicators, but they 
are not as large and not as consistent across indicators as are those with school insider initiation 
(district personnel range =.03 to .20, Mdn = .08; researchers range = .00 to .13, Mdn = .08). 
Appendix Table H5.3 also shows a consistently positive association between program quality and 
local development of the activity, but these associations are of very small magnitude. 
Researcher-developed programs (although relatively rare compared with non-researcher- 
developed programs) also have a slight advantage on five of the eight indicators of quality. 

Another indicator of integration into normal school operations is the extent to which the 
program was selected after a deliberate attempt to seek information about what would work in 
the school. Programs or activities selected after a more extensive information search are, as 
expected, implemented in a higher quality fashion. Correlations with seven of the eight quality 
indicators with the number of different sources of information used to select an activity are 
greater than zero and in the positive direction. (Appendix Table H5.3 shows that the range of 
correlations is from -.01 to .18, Mdn = .10). Activities selected after using many sources of 
information are especially likely to incorporate “best practices” with respect to content. 

Programs whose coordinator’s jobs are more dedicated to the program, whose coordinators 
work full-time in the school, which are not delivered by volunteers, and that are part of the 
regular school program were also hypothesized to be better-implemented. These hypotheses are 
strongly supported. Of the 32 relevant correlations, only three are not in the expected direction 
and the confidence intervals for these include zero. Twenty-three (23) of the relevant 
correlations are statistically significant. Although most of these correlations are in the small 
range, a few are of moderate magnitude (details are in Appendix Table H5.1). 

Table 5.4 shows selected indicators of program quality as a function of those indicators of 
integration into normal school operations that best predicted quality. 

The evidence supports the contention that one way to improve the quality of implementation 
of prevention programs is to ensure that they are better integrated into normal school operations. 
More extensive local planning and involvement in decisions about what to implement, use of 
regular school staff as implementors (particularly when a larger portion of their regular job is 
dedicated to the activity), and incorporation of the activity as a regular part of the school’s 
program all predict higher quality implementation. 
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Table 5.4 

Activity Quality by Indicators of Integration Into Normal School Operations 

Proportion “best Proportion "best Proportion students 

practices” used — practices” used — Frequency of exposed or 
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Feasibility. We asked program coordinators to indicate to what extent their activity 
depended upon special arrangements or materials not usually readily available in schools. We 
expected that the number of such “obstacles” named would predict poorer implementation. The 
data produce mixed results relevant to this hypothesis. As expected, more obstacles are 
associated with somewhat less frequent staff participation and less frequent program operation. 
But more obstacles are also associated with somewhat higher scores on the “level of use” scale, 
and a higher proportion of best practice methods used. All correlations with the number of 
obstacles were small (ranging from -.10 to .07), and only half were significantly different from 
zero (details are in Appendix Table H5.1). 

The time of day when the program or activities are conducted was also expected to predict 
its degree of implementation. The data reveal that the intensity, frequency of staff participation, 
and proportion of students exposed are higher for activities that take place during the school day, 
although the quality of the content and methods are not necessarily higher for activities that 
operate at this time. Programs or activities taking place before the school day also get higher 
staff participation and regularity in operation. The data are mixed for after-school programs: two 
indicators of staff participation are slightly higher for after school programs, but a significantly 
smaller proportion of students are exposed, the intensity is lower, and the quality of the program 
content is lower in these programs than in programs run at other times. Programs run in the 
evening and at night are also less intense. Details are shown in Appendix Table H5. 1 . Table 5.3 
shows mean scores for selected indicators of program quality as a function of those times of day 
most associated with frequency of staff participation and program intensity. 

Summary. We found substantial support for the following hypotheses: 

1. Greater levels of conscientiousness and past accomplishments on the part of the program 
coordinator are associated with better quality of program implementation. The associations are 
small, however. 

2. Better integration of the activity into normal school operations is associated with higher 
quality programming. More extensive local planning and involvement in decisions about what to 
implement, use of regular school staff as implementors (particularly when a larger portion of 
their regular job is dedicated to the activity), and incorporation of the activity as a regular part of 
the school’s program are associated with higher quality implementation. 

3. Greater organizational support is associated with higher quality implementation. More 
training, higher quality training, more supervision, and higher levels of principal support for the 
prevention activity should increase the quality of implementation. 

4. Greater standardization of program materials and methods is associated with higher 
quality implementation. 
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Support for the importance of perceived school amenability to program implementation, 
budget control and program feasibility was mixed. The importance of budget control and 
certainty of continued funding was not supported, and the quality of programming is generally 
not enhanced by the absence of unusual program requirements. Programs run during the school 
day or just before school are, however, generally of higher intensity than programs run at other 
times. 

Other Program or Activity Characteristics 

Several additional characteristics of prevention programs measured in the program 
coordinator surveys were not specifically hypothesized to predict program quality and intensity 
but are of interest. These characteristics are discussed now. 

Source of ' funding. Correlations between the activity quality indicators and the specific 
sources of funding for the activity are not large, but some interesting patterns emerge. Activities 
which are “institutionalized” in the sense that they are funded by their own school district are 
delivered with slightly greater intensity, greater frequency, and a higher level of use than 
programs that are not funded in this manner, but they are not necessarily of higher quality in 
terms of their content and methods. Programs funded through the Safe and Drug Free Schools 
and Communities Program (a Federal program that distributes approximately a half billion 
dollars per year to schools for prevention activities) make more use of best practices with respect 
to content but SDFS funding has only small (.08 or less in absolute value) correlations with other 
indicators of program quality. 5 These and other results are presented in detail in Appendix Table 
H5.3. Correlations are generally small between sources of funding and indicators of quality. 

Cultural appropriateness. As the country’s school population has become increasingly 
diverse, and as sensitivity about insensitive and inappropriate curricular materials or 
interpersonal approaches has increased in recent years, many educators and prevention workers 
have become increasingly concerned with the “cultural appropriateness” of prevention materials 
and methods. In surveys, we asked activity coordinators to indicate whether their activities were 
specially tailored for a particular group (e.g., females, African Americans, gay or lesbian youths); 
intended to foster understanding, respect, or appreciation for the diverse needs, traditions, or 
situations of particular groups (e.g., males, persons of different ethnic origins, persons of 
different religion); or used materials or methods culturally appropriate for the students served. 
Programs or activities that are specially designed to foster understanding for persons of different 
ethnic origins, cultural heritages, languages, etc. are better implemented in some ways than 
programs not so tailored, but the associations are very small, ranging from -.03 to .14, Mdn - 



5 One speculation is that this pattern may result because SDFS-funded activities are more likely to 
make use of canned programs than activities without this source of funding. For example, of 
activities for which SDFS support was reported, 9% are D.A.R.E. programs, which use a higher 
proportion of best practices with respect to content, but which has failed to incorporate best 
practices with respect to method. 
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.04. The evidence is inconsistent for programs that are specially tailored for at least one of eight 
different groups listed (e.g., African Americans, Asian Americans, males). These programs 
make use of a slightly higher proportion of best practices with respect to methods, but they are 
less frequently operated than other programs. Correlations range from -.07 to .08 with the 
measures of quality, Mdn = .02. The program coordinator’s perception of the program activity as 
“culturally appropriate” is slightly positively related to most indicators of program intensity and 
use, but not with the use of research-based content and methods. These correlations are also 
small. All the correlations are displayed in Appendix Table H5.1. 

Characteristics of the population. Data were scrutinized to learn whether there were 
correlations between indicators of program quality and the specific groups targeted by the 
program or activity. For the most part, these correlations suggest that the population targeted is 
not much associated with the quality of the program. An exception is the expected observation 
that universal programs (programs directed at no special group) serve higher proportions of 
students than do targeted programs. A second (fortunate) exception is that activities directed at 
problem students or students about to be expelled involve lower proportions of students. Aside 
from these exceptions, the correlations are small and inconsistent across the different indicators 
of program quality. Details are presented in Appendix Table H5.7. 

Activity objectives. Activity coordinators were presented with a list of potential activity 
objectives and asked to identify those addressed by the program or activity. This allowed an 
examination of the relations between specific activity objectives and activity quality, as well as 
an examination of the relation between the breadth of an activityls objectives (the number of 
different objectives identified) and program quality. The most striking finding is that the breadth 
of the program’s objectives is significantly positively correlated with seven of the eight indicators 
of program quality and quantity, and for best practices with respect to content the correlation is 
large. The correlations range from .00 to .43, Mdn = .10. Correlations of program quality with 
the various specific objectives are generally slightly positive, and are generally moderately 
positive with the indicator of best practices with respect to content (two exceptions being 
programs targeting religious beliefs and parental supervision). The correlations between best 
practices (content) and the thirteen specific objectives range from - .07 to .30, Mdn = .22. 
Programs targeting social skills and competencies as objectives do not have a very favorable 
pattern of correlations with the quality indicators. This type of program has been identified in 
efficacy research as one of the potentially most effective in terms of its effects on problem 
behavior. The present results imply that as applied in schools such programs typically do have a 
larger proportion of best practices with respect to content (by definition), but that they less often 
use best practices with respect to methods, are less intense, and expose a smaller proportion of 
students to the activity than do programs without social skills objectives. Details of the relations 
between activity objectives and the eight quality indicators are presented in Appendix Table 
H5.8. 

Activity content. Correlations of measures of program quality with specific activity 
categories (expressed as dummy variables) are simply a different way of expressing the 
associations between program type and quality of implementation discussed in Chapter 4 (and 
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summarized in Table 4.7). Nevertheless, such a table of correlations is presented in Appendix 
Table H5.9. That table also shows correlations with the multi-component nature of a program or 
activity and whether it is a packaged program. (Recall that a multi-component program is one 
that was identified by the school administrator responding to the principal survey for program 
identification as belonging to multiple categories, and that a “packaged” program is an activity 
recognizable as a widely marketed product such as D.A.R.E. or Assertive Discipline.) Although 
we found evidence (just described) that programs targeting more objectives are implemented 
with higher quality than are programs with narrower objectives, we also see evidence that when a 
program activity is one component in a larger activity that contains several different types of 
activities, its implementation quality may suffer. The correlations between multi-component 
status and the eight quality indicators are small and only reach statistical significance for two of 
the eight indicators - in both cases negatively correlated with quality. Correlations range from 
- .06 to .02, Mdn = - .04. These results argue against claims that multi-modal programming will 
be more effective. Although we have no data on the effectiveness of these programs for 
achieving their objectives, the correlations (presented in Appendix Table H5.9) suggest that 
multi-component programs are not particularly well implemented in practice. 

The pattern of results is similar for packaged programs. When all packaged programs are 
grouped together, the evidence suggests that they are not as well implemented as home-grown or 
less well-known programs. The correlations (Appendix Table H5.9) are small, however. In the 
following sections, we examine packaged and multi-component programs more closely. 

Table 5 .5 provides a summary of those hypotheses that were supported by examination of 
the activity-level data, and it provides a summary of the most striking findings from the 
exploratory data analyses. 

A Closer Look at Specific “Packaged” Programs 

Aggregating all eleven “packaged” or “canned” programs into one category as was done in 
the examination above may disguise important differences among them. In this section we 
describe the quality of the two canned programs that were over-sampled — D.A.R.E. and peer 
mediation. Among the 1 ,087 packaged programs that principals named on the Phase 1 survey 
were 305 D.A.R.E. and 308 peer mediation programs. These were sampled with probability equal 
to 1.0 in Phase 2. From these sampled programs, 174 (57%) and 142 (46%) completed Activity 
Questionnaires were returned. D.A.R.E. programs were described in these returned 
questionnaires primarily as prevention curricula (48%) and uses of external personnel resources 
(38%), and less often as programs to improve the culture or climate of the school (9%). Peer 
mediation programs were described primarily as programs to involve youths in discipline (54%). 
Peer mediation activities were listed under a number of other categories as well (e.g., as 
prevention curricula or counseling programs). 
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Table 5.5 

Summary of Activity-Level Correlates of Quality of Implementation 



Hypotheses supported by the data 

1 . Greater levels of program coordinator conscientiousness and coordinator’s record of past 
accomplishments are slightly associated with better quality of program implementation. 

2. Better integration of the activity into normal school operations is associated with higher 
quality implementation. More extensive local planning and involvement in decisions 
about what to implement, use of regular school staff as implementors (particularly when a 
larger portion of their regular job is dedicated to the activity), and incorporation of the 
activity as a regular part of the school’s program are associated with higher quality 
implementation. 

3. Greater organizational support is associated with higher quality implementation. 
Specifically, more training, higher quality training, more supervision, and higher levels of 
principal support for the prevention activity are associated with higher quality of 
implementation. 

4. Greater standardization of program materials and methods is associated with higher 
quality implementation. This means that activities for which there is a manual, written 
descriptions of specific activities or methods to be carried out, prepared materials such as 
visual aids, lists of materials, or reproducible materials are better implemented. 

Patterns revealed by the data (although not hypothesized) 

5. Activities which are “institutionalized” in the sense that they are funded by their own 
school district are delivered with slightly greater intensity, greater frequency, and a higher 
level of use than programs that are not funded in this manner, but they are not necessarily 
of higher quality in terms of their content and methods. 

6. The breadth of programs’ objectives is positively associated with program quality and 
quantity. 

7. Multi-component programs (i.e., those involving several different categories of preventive 
activity) are not as well implemented as single category programs, although the 
association is small. 

8. “Packaged” or “canned” programs as a group are not as well implemented as “home- 
grown” programs, although the association is small. 



When the quality of these packaged programs is compared with the quality of all other 
prevention activities, we see very little difference: the average percentage of quality dimensions 
judged adequate is 55% for D.A.R.E. programs compared with 57% for all other programs. 
Similarly, the average percentage of quality dimensions in peer mediation programs judged 
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